Do you have an enclosure for 3.5" S-ATA HDDs to see if you can see it while running Mac OS X from the original HDD?
What exact model of 500GB HDD did you buy?
Btw, the iMac can have HDDs bigger than 500GB, iFixIt even links to 1TB HDDs in
its HDD replacement guide.