Could be the drive's controller board not "interacting right" with the Mini.
I've seen similar behavior with some drives on my own 2018 Mini.
This seems to be more likely on drives that don't have a physical on/off switch, but instead rely on the "presence of bus power" to turn them on/off (via some kind of internal switch or controller).
Solution (well, for me, it's "the solution", though others might not agree):
Once you dismount the drive, power it down -- this means pulling its power block out of the power strip if need be.
If it has a power switch, turn it off.
Or -- physically disconnect the USB cable.
Rant:
Whatever happened to "on/off" switches on peripherals?
It was easy to just reach back and "turn it off" when you didn't need it.
But they're mostly gone these days.