Aside from raw tech spec comparisons, here are some practical thoughts.
I'm in the process of setting up a Mini to (a) replace the sawtooth G4 I have running OSX Server (for AFP, Web and email) and (b) to act as a music library, so all our CDs can be banished to the basement. I bought it refurb'd and sold my quicksilver G4 to offset costs. It still only has the base 512Mb RAM shipped, as I'll buy cheaper third-party chips independently. There's a lot of beachballs and pauses if I play more than Music (iTunes), so budget for a healthy memory upgrade. I plan to go the max of 2GB.
So far, it works great--even hooked up to my 15 year old CRT. The DVD controls are pretty simple, so we'll keep our dedicated player for a while yet. For iPhoto, plan which mac will be the primary database of pictures, and consider how photo cards will be dumped
For maintenance and set-up, I use Timbuktu to remotely control the box from my MBP, so the CRT resolution (i.e. lack thereof) is not a great issue. My plan is to keep it running for server apps in the background, and use Front Row via remote for media apps. I'll buy the bluetooth keyboard/mouse when I get a LCD TV.
The thing to remember is about the Mini as a media centre is that Front Row occasionally times out, or iTunes pops up a dialog window, or Software Update will need to be run, etc. In other words, there will be an ongoing need to use the device "as a computer", whereas the

TV will likely maintain the look and feel of a consumer device throughout it's operation. Using the Mini as a media PC runs the risk of your wife/girlfriend/buddy shouting "Hey, the TV is stuck!".
That said, I recommend the Mini experience simply for the expandability potential (PVR, OSX apps, etc).