Remember, that the iMac G5 have RAM with a slower FSB (533 MHz), while the Core Duo's have 667 MHz. Also the fact that they upgraded from PC-4200 to PC2-5300 makes a difference. Remember, that the processor is how fast you do stuff, and RAM is how much you can do at a time. I would say that you could go with 1.5 GB, and have okay performance, When you get a universal thingy, you can run it very fast with 1.5 GB. Just my two cents!
(BTW, if you have 512 now, it would have been much cheaper to buy 2 GB from start (+$300). But now, you can buy a 1 GB-dimm for $300 and have 1.5 GB
)