Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Yes, if the only devices you have are an iMac and an iPhone, and the the Mac is switched off, only your iPhone will show up in the Find My iPhone.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.