I never use the Mac remotes, and I use a Logitech Harmony remote for my home theater setup. including TV. I have 3 Apple remotes at home (iMac, MB, and TV, and one at work (iMac). None of them ever get used.
I have 2, I left the wrapper on my iMac remote since I don't use it and it sits on the foot of the iMac. The Apple TV remote is usually on the coffee table next to the tv remote or on the mantle.
Plus both are paired to their specific device so the Apple TV remote won't activate Front Row on the iMac and vice versa.
I keep the one I'm not using in my coffee table drawer. I did get them mixed up last week when they both randomly unpaired themselves, but otherwise it's a system that works out well.
My problem is that I have an Apple TV remote and the HiFi remote. I used the HiFi's optical input from the AppleTV for my "home theatre" (I'm in an apartment so I don't want to be 'that guy' who cranks 5.1 with a huge subwoofer all night). Since they're both on the coffee table they got mixed up a lot. I solved this problem by MacGyver'ing my AppleTV remote on the underside of the TV remote to make one mega-remote. The free apple remote is then the one for the speaker. Not idea, but it works.
We have 4 Apple remotes in the house, each paired to their own Mac (2 Mac mini's, a MacBook Pro and a MacBook). I've had to write on the back of each remote to which Mac they belong to.
I can tell because my DOg chewed up the one I have paired with the AppleTV - the other two dont look like chewed up pieces of gum But it still works great