Sacrilegious I know, but you can pick up second-hand PC DIMMs quite cheaply now, pulled out of old Pentium 3s. PC100 and 133 have worked fine for me. You can get away with 512, but it does struggle a bit. If it has one 128 DIMM in it already I'd recommend getting a 512 to go with it. If it has two 64s, go for 2x512.
Upgrading the HD will make a difference too, not just in terms of working space, or the fact that X likes to have 8GB free, but the newer drives are a lot quieter than the old graunching IBMs the iMacs came with, and generally faster too.
Not that imagine you would, but don't go higher than a 120GB drive because the iMac can't see any more than that anyway (137GB actually, if you put a 160 in)