OK, I just got back from CompUSA where I had a chance to use a new 20" side by side with an old 24". The lighting was about as bad as can be for glossy displays; row after row of 1000w halogen bulbs right behind me, and long, bright floresent bulbs above me. I played with iPhoto/iMovie/iTunes a bit, then watched an HD Apple trailer (Bourne Ultimatum) on both.
The glare was obvious on the 20", but only the 1000w halogen bulbs. If you have a bright point light source right behind you, it could be a problem, regular light wouldn't bother me a bit though. Even with that, the 20" was BETTER than the 24", for a number of reasons.
1) The colors were WAY more vivid... looked great.
2) The refresh/update speed was clearly better - the action scenes in the trailer were much sharper.
3) The glare was actually more distracting on the 24" matte display. The very bright lights caused a large blownout area on the 24" that made about 1/3 of the screen look like crap. In contrast, the glass on the 20" reflected a point (well, a large point to be fair) of light, BUT I was able to easily look past/through it and focus on the screen and the end result was much better.
4) I didn't notice any dithering or color shift, even from moderate angles (say 100-120 degrees, which is pretty much all the usable viewing cone you should need if you aren't using it for large groups. Even from near 180 degrees the color shift was minimal, if noticable).
Now, I can understand the issue of you are doing serious photo editing and need 100% color accuracy, but for the rest of us... well, let's just say I am EXTREMELY excited to get my new iMac ASAP!
