I'm a huge game player, but when I've got time to play a game I prefer to do so on a console or a PC as thats a rich, immersive experience and with controls that are optimized for gaming.
Conversely, casual games are meant to kill time while waiting for real life to happen...doctors offices, sitting on the train, airplane, etc. Since I'm not a Murse-wearing fancy-boy, I don't have my iPad with me all the time and for the casual games, I find the iPhone to provide a far better gaming experience. Yes, the screen is small, but the kinds of games I enjoy playing (Solitaire, lightweight shoot-em-ups, etc.) are optimized for a small handheld platform such as the iPhone. The iPad, while no doubt providing better visuals (although I'd say the retina display of the iP4 makes that a debatable topic), is awkward to hold for extended gaming sessions while on the move. And if I'm not on the move, I'm back to the aforementioned "real" gaming rigs hooked up to a big plasma and surround sound system.
So while I do keep a few games on the iPad for grins and giggles, nope...I find it poorly suited as a "great gaming rig." Its too big for casual games, too weak for dedicated big-screen gaming experiences.