If you buy a refurb, they're held to the same standards as new, and have the same warranty. You can buy applecare on it just as easily as a new one. The only difference between buying a new one at an apple retail store and a refurb online is that you have to wait two or three days for the refurb to be shipped.
The refurbs usually has a refurb sticker on the back, since it is essentially remanufactured. Otherwise, when you enter the serial number, apple support may tell you that you're not eligible to buy applecare since the computer was manufactured over a year ago.
And when I sold my g5 powermac, two people seriously looked at it. The first did turn his nose up a little bit at first at it originally being a refurb FOUR YEARS AGO. The other didn't care and bought it instantly.
I view it as buying a one year old car; the car still has the warranty, the bugs have been identified and worked out in it, and you get essentially a new car for 10-20% off. Except there's less of a difference between buying a new imac and an identical refurb.
I bought a new in box white imac last month, and it is closer to the two year old car analogy. It's still a new computer.