I'm coming in late to the game, but here's my personal take on it. I already had a Canon EOS 30 (film SLR), with three lenses: the 50mm f/1.8, the 75-300mm USM (third model), and the 28-105mm USM, so going with a Canon body was pretty much a no brainer.
When I went shopping, I knew pretty much what I wanted to do with it: wildlife photography. This means long lenses. I was going to use the 75-300 until I could afford something better, but one shot was enough to persuade me that it's a piece of junk (harder to tell on film, but digital really showed up the flaws. Hooray for a generous co-worker that has a 70-200 f/2.8 IS gathering dust ...) Anyway.
I tried both out in the shop, and had no particular problem with the feel of either. The girl showing me the bodies (who happened to be my cousin ... what a strange coincidence ...

) was pushing me towards the 20D, but I couldn't see any good reason for me to do so. Eventually, on the advice of another friend, I asked her to get down the 100-400mm L series lens (which weighs around 1.3 kg), and put it on the two bodies.
That sold me on the 20D: the feel of that lens on the 350D (the Australian version of the Rebel XT) was not a good one. It felt almost -- not quite, but almost -- like it was threatening to fall off; the camera body didn't counterbalance the weight of the lens ... it just felt plain wrong, and very fragile. On the 20D, though, it felt pretty solid.
There's nothing wrong with the 350D. If you're not going to be particularly serious with your photography, it will do the job, and do it well. But if you want a body that will last a long time, and which will stand up a bit better to the rigours of a long day of shooting, I'd recommend the 20D.
In short: know what you intend to do with the camera, and what the limitations are. Build quality may or may not be worth the extra money to you; the other advantages of the 20D are really just icing on the cake, at least for me. I'm certainly not going to say "don't buy the 350D", because I don't know where you intend to take your photography; the 20D may well be overkill for your needs. I can't judge that. All I can say is "this is what I went through, and this is why I chose the model I chose", and hope it's of use to you.
Good luck ... and if anybody tries to sell you a Canon 75-300mm lens, laugh in their face. It's not worth the money.
Anybody want to buy mine?

(I'd post the pic that shows why I'm so down on that particular lens, but it's about 1.7 MB. Too big for most free web hosts ... any suggestions? I
don't want to reduce it, as that would reduce the artifacts ...)