There are at least three elements to working with photos: 1) storage and management, 2) developing and adjustment (white balance, contrast, tint, etc.), and 3) editing (changing actual pixels) (I'll leave publishing and printing out)
iPhoto has a) very simple, but easy to use, storage and management, b) simple and not very robust developing tools and c) very few editing tools (spot and red eye)
Aperture has a) great storage and management, b) powerful developing tools, and c) a couple of good editing tools (red eye, spot and patch, vignette/de-vignette)
Photoshop (alone, without bridge or Lightroom) has a) no storage and management, b) the ability to do anything you want in developing and adjusting (but layers really do require a learning curve -- powerful, yes. Easy, not at first) and c) a full suite of editing tools (add text, change the background, but, again, using layers).
Most people getting started need management first, with some basic development tools. If you are at all particular about how your pictures look, though, you start needing better developing tools very quickly. So, iPhoto is a great starting point, but Ap will really help. Not everyone will use Photoshop or want to use Photoshop. But, if you do want to do serious editing, you can combine PS with either iPhoto or Aperture. If you want to really learn Photoshop, you might not even need Ap.
So, which one to use for organization and development? I would say iPhoto if your picture volume per week is fairly low and you are shooting jpeg. Aperture if you are shooting RAW and/or high volumes of pictures.
Having said all that, PS Elements 6 is coming out soon (end of March!), so don't buy PS Elements 4! Go on the Adobe site and take a look at PS 6. It's got some really cool features for amateurs and hobbyists/home types. See for example
http://www.adobe.com/products/photoshopelmac/features/#item2 or view the videos at
http://www.adobe.com/products/photoshopelmac/training.html#seehow. You might want to hold off on your decision for 3 weeks or so, and see what PSE6 looks like, and try the demo, before final decision.