I agree with you. I've been buying quite a few of my songs from Amazon rather than iTunes lately.
If anyone from Apple is listening here, I'll mention that I buy from Apple when they have extra "Apple-only" incentives, like extra (bonus) tracks or free movies included with an album purchase on a pre-order. Those are the things I love to see from Apple!
It's easier and I just don't like Amazon just yet, the features of buying there are good, but the conveniences still don't beat iTunes. You could buy from there, and load into iTunes I guess. As someone already said, preleases and singles. Amazon doesn't have iTunes' vast pool of music selection.
I'm hooked on Amazon's cheap albums of the day/week, and it's not so trouble at all to transfer songs from Amazon into your iTunes library - it does it as soon as the downloader finishes.
I buy Amazon in most cases since it's almost always cheaper than Apple's iTunes Plus. I refuse to buy anything DRM. It works flawlessly for me, I have no complaints.
I've been trying to look around amazon but it's just confusing to find songs. Maybe I'm not used to it. Plus I like the ability to go to a store and pick up an iTunes gift card instead of having to use my credit card.
I've known about Amazon since they've been selling MP3s but I still prefer the iTunes store for many reasons:
Convenience
Quality - I know I can get a refund quickly if there is a defect. I got my $99 the day the Complete Led Zeppelin was released because the gapless playback wouldn't work on the first few albums
Availability
Everything I am interested in, iTunes has it. A very recent example is U2 Live In Paris. Itunes has it, Amazon doesn't. I bought it a few days ago in iTunes. Amazon loses the sale.