So I fixed my issue that looked like this by removing my Google / mail accounts in System Preferences | Internet Accounts (as well as a rogue-looking 'Contacts' entry in the same list). I've re-added the accounts without the crashes restarting.
And while this seems bizarre (my crashes also appeared GFX related), I arrived at this conclusion logically. I installed a fresh install of Catalina next to my main OS.
I couldn't reproduce the crash on the fresh install. I re-added apps and services one at a time and left the machine for a little while after adding each thing. If the OS session was still alive when I returned, that app or service wasn't the issue. The crashing only started when I enabled my Google / mail accounts in that Preferences panel. The crashing only stopped when I deleted them from that list (disabling again didn't help).
It's worth noting that in my case, I was able to reliably reproduce the crash by enabling 'Prevent computer from sleeping automatically when the display is off', and setting 'Turn display off after' to 1 min in Energy Saver and just leaving the machine alone. With this configuration the Mac OS session would last at most 10 mins left untouched before Kernel Panic (sometimes just BAD MAGIC, but also some with other errors like the one
@BillyMacAttack mentioned).
I've performed the same dance on my main OS (removed the accounts then re-added them), and haven't experienced a KP in about 1 day, even with the aforementioned Energy Saver settings.
I hope this helps somebody.