Biggest frustration with the game so far is loading time. Often when you transition between locations (i.e. open the main gate to leave a city) there is a transition screen, which normally has a picture from the game (like an Axe or something) and some game information ("Skyrim was initially colonised by blah"; "Holding down the power button will make your shouts more powerful"). My computer is a few years old, and so it might be that my hardware isn't up to scratch, but you can sometimes end up sitting there for 40 seconds while the new environment loads. I've taken to using this time to go and get snacks / boil the jug / hang out washing etc.
It just becomes particularly frustrating when there are two of these transitions side by side. For example, when I leave my house in Markath (Vintrel Hall), there is a transition (into Markath). Then if I want to fast travel to somewhere there's another transition. So two transitions within a few seconds of each other, meaning I can be waiting for a couple of minutes.
Fast Travel is useful though. The idea is that if you'e already discovered a location, you can simply open up your map and click on that location to travel there. Time passes in game, but it saves you to boredom of having to actually spend 20 minutes travelling normally. I find I spend a lot of time fast travelling between the main cities (all of which you can initially discover by catching a coach).
The other think that I've found is that there are just so many quests! It's at the point where I am actively avoiding getting given any more quests, as my journal is so full. Some Orc stronghold wanted me to bring them a couple of ingredients to help break their curse. Okay, no problem, I'll keep my eye open for them. But then they want me to help them go to a cave and beat up some monsters... bugger off, I've got better things to do with my time than solve all your problems.
Skills cap out at 100. I got my Blacksmithing up to that by basically making all the armour in the world. I'd go out of my way to kill animals, take their hides, tan them, and make them into armour, which I'd then sell. Blacksmithing skill increases and I get money. Win!
I also found that it starts to take way longer to level up, mainly because the skills you use the most get really high, and then you need to do so much more to increase them. With Blacksmithing already capped out, my other main skills (One Handed Weapons 89, Heavy Armour 87, Block 71) require me to hit or be hit maybe 100 times to go up. Given I normally kill opponents in two hits, it's about 25 fights to increase one of these. The temptation is to dilute my characters focus and start working to increase my lower skills. Already I've fought some easier fights using a two handed weapon (skill currently at 25) just to get some quick increases to get me closer to leveling up.
But, despite all this, I'm still loving it. I'm sure I'll get back to the main quest eventually, once I've learned a few more Dragon Shouts...