How much RAM do you have?
I now have 16 gb (I did it myself, I found the available tutorials on the net to be very helpful, so I was able to do it safely, even though I am not otherwise experienced with such things), but when I did my tests and the above observations, I only had the standard 8 gb of RAM.
I found that my geekbench scores got a little bit better after I went from 8 to 16 Gb:
With 8 GB it was 5183 Single-Core and 20084 Multi-Core.
With 16 GB it is now 5353 and 21911.
The felt difference in daily use (8 versus 16 GB) is slim. When I deliberately tried to get my machine to swap and develop "yellow" memory pressure, I was able to do so at some point (boatloads of heavy Safari tabs and 1080p Youtube.) And in normal daily use, my Mac Mini with 8 GB did some minor swapping. But none of this was by any means slowing me down.
Now with 16 GB, the machine never swaps in my usual, daily use. Some memory compression, but that is how it is supposed to be, even with lots of RAM.
I am MIGHTILY pleased with the Mac Mini 2018 (i5, 256gb SSD) that I have.
EDIT/addition: And I am VERY pleased with how cool and silent the i5 is running. This was a concern of mine before buying, because it sounded on the forums that only the i3 would be silent all the time.
Daily use, including playing Youtube 1080p fullscreen on a 4k monitor: the fan is running at baseline-speed, which (and I am picky!) means it is silent. Even at night when everything is quiet I would not be able to tell (from where I sit) whether the Mac Mini is running or not.
And the temperatures (CPU) are mostly in the 40ies (Celsius), dipping into the high 30ies, or peaking into low 50ies while playing 1080p 50fps fullscreen.
Case temperature as I am sitting and typing this: 31.5 deg. Celsius / 89 Fahrenheit (at 24 deg. Celsius room temperature). Which means the casing is cooler than body temperature. Nice!