Really? That's what I thought should be really easy to do BUT ATV2 does not want to connect. It sees the AEBS but just won't connect. I've tried everything. Made a new network and everything. WTF. So frustrated. I even tried switching to my Airport Express incase it was the Airport Extreme. Both had blinking yellow lights (assuming because there's no internet). My iPhone, MacBook and iPad all see any of the networks but the ATV won't connect to either. * And since we're on that topic of Airplay, my iPhone was connect to my Airport Express but the Airplay option wasn't on my iPhone. Nuts.
Are you using some special settings in the Airport Base Station Utility?
I should also not that my MacBook Airport Utility can find the AEBS when scanning. ATV2 sees it but cant' connect. I connected the AEBS to my MacBook using a LAN cable and obviously it saw it. That's how I was able to change settings etc.
Something's up.