I'd strongly suggest maxing it to 8gb.
I've found Yosemite+ with less than that to be somewhat uncomfortable...
BTW, an SSD can really make it scream. Folks are consistently amazed at how fast my Mini 1,1 feels(although it chokes if you really start beating on it) and that has a lot to do with it...even though I did run it for a while with a 2ghz T7200 Core2Duo and now have a 2.3ghz T7600.
Yes, 8 GB. It's about $65 US for that these days, which is OK. Plus a 120 GB SSD is only about $35 to $60, so total cost is $100-$125, for HUGE upgrade.
Micky, With those upgrades, El Capitan will run quite well. Even High Sierra will be decent, but High Sierra takes a bit more effort to get installed and some of those old 2009 Mac minis won't have functional WiFi in High Sierra (although Ethernet works fine).
For a geek with an inclination to meddle taking replacing the HDD with an SSD, plus more RAM than the Apple advised maximum, and embracing the challenge to install a more recent OS than officially supports older computers, to get them to "scream" can provide immense satisfaction.
For the average Joe or Jill, who just wants to do some relatively undemanding stuff, just a dust out every three years or so, and upgrade / update within officially supported specs and OS, can keep an older Mac good for day to day use for ten or a dozen years, maybe more, with minimal hassle.
Each to their own, but I fall into the average Joe camp.
I got the first Mac Mini in 2005 because I wanted something that was easily occasionally portable. When the HDD and power supply failed in 2009, my situation and requirements had not changed. I reckoned that replacement was more cost effective than repair..... a good decision as it turned out.
My needs have changed a bit now. Going ahead, as a teacher, I'll be using a lot more computer based material, so need something I can take to work most days. I'm going to get a MacBook Air, mainly for work, and continue to use the Mini as my main machine at home for another couple or three years.
When some part of the Mini does fail, as sooner or later it must, replacement will again probably be more cost effect than repair from the point of an average Joe. A geek, on the other hand, may see it as a project to be improved and pushed to see what is possible.