People who can't afford Apple products should just look elsewhere for cheaper products that will do the same or more for less money. Think of the Mac Mini as a Porsche Boxter. It's an entry level Porsche but that doesn't make it cheap or an entry level car...
At it's current price point the Mac Mini is a pretty reasonable machine, not just as a Mac; it was even better when it came out and the various components were more recent. The premium you're paying may be partly for the brand, but it's also for a neat, superbly engineered little machine; for the same money you can get bigger, more powerful computers, but not many that offer the same performance in a similar package.
But you're talking about driving up the cost for no clear reason, when it's perfectly feasible to just continue to offer the customers the choice. If you want to insist on using the Porsche Boxster as an example, you're talking along the lines of dropping the basic model in favour of forcing people to only by the GTS; it's faster sure, but also more expensive, and yet Porsche can clearly see the advantage of offering the choice of the more modest, more affordable model.
I tried my mini with its 5400 stock drive. MOLASSES. I would give up computers if I had to use it.
Doing what? In my household we have a Mac Pro (mine, uses an SSD to speed up the OS), a low end iMac, and two low-end Mac Minis with 4gb RAM (one would have had stock 2gb otherwise, at the time). The family members using the Mac Minis are perfectly happy with the speed of the machines themselves, in fact the main speed issues are games not running well but that's usually the fault of the GPUs.
I also use the Mac Minis any time I need to make sure some updates are applied, or if my own machine isn't switched on at the time, or if I need to print something (since people keeping switching the damned thing off so there's no point in my using printer sharing); my point being that I use the Mac Minis fairly regularly, and I find they perform just fine. Thanks to being such quiet, power efficient machines it's a simple matter to put them to sleep instead of shutting down, which means they're very quick from being woken up to logging in and using them.
Granted the hard-drives aren't the greatest in the world, but there are 7200rpm drives Apple could use instead, or even hybrid SSHDs, both of which could be used without having to drive up the cost of Apple's entry level machine
for no reason. If a customer wants faster, then they're still free to switch to an SSD if they want to. I find it appalling than you'd see customers forced to take a more expensive option that they may only see relatively small benefits from (unlike laptop users who see multiple benefits), or who may even be inconvenienced if the lower capacities cause them to run out of space. In a few years as SSDs continue to go down in cost, sure, but now is still not time for SSDs to become the norm for desktops.