Ballmer's problem is that he is neither a visionary nor a good leader. He seems to have very little interest in technology in general, and doesn't really know or care, where the industry is going. On top of that he, and other senior leadership at Microsoft, still seem to be under the impression that it is still 1997 and Microsoft's only real competitors come from inside Microsoft. There are countless stories about how Microsoft execs constantly snipe each other and jostle for "ownership" of various products. The result is just an inconsistent mess in both their product lineups and within the products themselves. Inconsistent GUIs, various software thats not compatible etc.
Microsoft needs a shakeup if they are going to survive. Whether Gates is willing or able to do so remains to be seen, but Ballmer should have been thrown out years ago.