Whilst it shouldn't make any difference for your average run of the mill UIKit orientated apps there is quite a significant difference between the devices where games and highly intensive graphical applications are concerned.
In order of most powerful to least powerful:-
iPod Touch 2G -> iPhone 3G -> iPhone -> iPod Touch
We have 3 apps out on the App Store at the minute and all of them run fine across all the devices as they are using mainly UIKit and CoreGraphics. We are currently making a game using OpenGL and we have noticed significant differences between the upper end and the lower end of the devices where performance is concerned