Hope I can help, but I was in exactly the same position as you. I was all geared up for a 20 incher; I really thought the 24 was overkill, especially after seeing the size of the chunky white 24 inch imacs in the previous generation!
However, as soon as I found that the screen in the 20 inch was inferior to the 24 inch in quality, this really got me questioning the 20 inch. I checked out the 24 inch regarding the size, and although big, it is certainly not ridiculous in size compared to its predecessor, and it is slightly smaller in dimensions too.
However, what really swung it for me was the quality of the screen. I do a lot of graphics work, and as I have a lot of people looking around my screen at photos, logos, video etc, the restricted viewing angle on the 20 inch, compared to the 24 really wasn't worth connsidering in the end. The 24 inch for me is the only way to go. I think you'll always regret getting the 20 inch over the 24....I know I would.