This game was, i think, based on the Doom engine since it played kinda similar but the one main difference is that You could get magic spells or something, there were mana vials that you could pick up and so forth.
AFAIK, they only made Heretic/Hexen for PC (but I didn't have a Mac back then, so I could be very very wrong).
You'd need a virtual machine running Windows to play...and AFAIK you can't get just the Doom engine, you'd have to get either Heretic or Hexen (which would include the Doom engine) as a standalone game.
Can I only run Doom, Heretic and whatever in the Doomsday Engine?
WHat If I want to play Wolfenstein Duke Nukem and others?