It's been done before, particularly on Sony players and phones.
And now I'm debating between a new 16 GB nano or a refurb previous gen. 8 GB touch. I'd prefer the larger capacity, and if I got the touch I'd keep my old nano (I like the small form factor), but even if I got the 16 GB nano, I doubt I'd be able to sell my old one for much. It's "used" condition is very evident, and it's now two generations old!
Touch it is, then?
I honestly don't know what to tell you. Touch has more features if you want games and apps.
If you just want music and want it in a spiffy, slim format. Like you say, the small form, then go for the touch.