There is no way of using the iPhone as a controller for a game that is being viewed on the big screen. The iPhone has ZERO tacticle feedback.
Check out Doom Classic if you want an example of "good" controls without tactile feedback. I played that game for a few minutes, and went into the options and turned off the "draw controls on screen" option so that you basically are using it without knowing where the buttons or virtual sticks are. It was really easy to control still, I had no problem knowing the layout that I selected. I could easily control two virtual sticks and shoot my gun and even switch weapons without even thinking about where the buttons were.
The key to achieving this is to layout "zones" on the screen. You essentially dedicate the left side of the screen to one joystick, and right side to another joystick. Then you have to corners be zoned for buttons. You can have the top right be shoot and the bottom right be jump for example.
Another option if your game only needs one joystick and a bunch of buttons, just have the left "zone" be the joystick and the right side have multiple zones. So you tap top right for one bottom, bottom right for another, and middle for a third button.
It gets harder as you add more buttons, so it's quite clear that games are never going to have really complex controls like you would get on a 360 or PS3 controller. It is never going to be as good as a real controller no matter what, but you can still work around that.
It still does stop you from making very cool games even if you have one virtual stick and 3 virtual buttons. It's not going to be about precision clearly. You just have to be a good developer and know your limits and work around them. But when you add in the possibilities of motion control with a gyroscope as well, it suddenly becomes less limited.
It is dumb to be elitist and say, "no tactile feedback, no physical buttons, there is no way that can be fun." There is plenty of ways it can be fun, it is just a matter of coming up with the right control scheme and understanding the limits.
That said, there is no way a virtual controller on an iPhone/iPod Touch would ever result in the big titles that you see on the 360 and PS3 but that mainly has to do with money and the fact that they are seen as big game platforms, but there is no reason new titles and games can't be made for iDevices. There is no reason they can't be very successful platforms.
There is also a lot more game genres to be explored, and iDevices have really proven that puzzle games and adventure games are extremely underrated and can make tons of money when paired with touch and motion based controls.
So yeah, don't be ignorant to the gaming possibilities of an Apple TV, but also don't be stupid and assume it would take on the 360/PS3 over night. Be rational, keep an open mind, and watch out for the potential successes.