Well, Apple doesn't allow competing music store apps on iPhone.
Also, at the Keynote, Apple said "Almost all of your existing iPhone apps will be compatible with iPad"- perhaps an out to kill some competing apps?
On the other hand, didn't Apple announce an in-App purchase process-- which they get 30% cut of anything purchased from within an App? Amazon will not be able to undercut Apple on ebook pricing if they have to give Apple 30%. So that would mean you have to buy an ebook on Amazon online, and then when you run the app, it would sync over (like with iPhone). Something tells me Apple will ban the new Amazon app.
Actually Apple supports many different music stores. Of interest to this topic is the Amazon MP3 store which directly imports into iTunes. This all works via public API methods for import into the iTunes library via the Amazon downloader. It is virtually seamless to buy from Amazon and have the tracks magically appear on your iPhone/iPod
In addition to traditional MP3 type stores, Apple has approved apps such as Rhapsody for subscription music and even allows streaming services such as Pandora. The only limitation to these apps is that at the moment you have to jailbreak to run them in the background, however, I bet this will change soon since Apple is a hardware company and doesn't make any money off of iTunes content sales (30% just barely covers bandwidth costs).
As discussed above, the iPhone already has a kindle App which in its current incarnation will work on day 1 on the iPad and will undoubtedly be updated for the iPad ASAP.
While apple does reject apps that duplicate functionality, these days apple is more interested in apps that use API's and GUI elements properly. By and large most rejections are actually due to these sorts of problems even if it seems off the surface to be due to iTunes store competition.