Interesting! I just checked, and it looks like Mail never recreated ~/Library/Caches/com.apple.email after I deleted it, which suggests it stores its caches somewhere else now. I did some digging, and here are some other folders you could try deleting:
1.
~/Library/Caches/CloudKit/com.apple.bird/com.apple.mail
2.
~/Library/Containers/com.apple.mail/Data/Library/Caches
3.
~/Library/Containers/com.apple.mail/Data/Library/Saved Application State
Basically, what seems to have changed is that Apple Mail is now "sandboxed" on macOS, for security purposes. If you're curious about "sandboxing" as a security practice, Apple has an article about it
here.
FWIW, the reason Mail.app doesn't appear in Time Machine probably has something to do with the read-only system volume, another security feature, in this case introduced in macOS 10.15 Catalina in 2019, which you can read about
here.
Because macOS and all of the core applications are now read-only, they can only be modified from the macOS Recovery system, which you can read about
here.
When you use Time Machine in the macOS Recovery system, instead of restoring individual files, you are restoring the entire internal disk drive to a previous state. In general, the fact that the system volume is read-only makes it effectively impossible for Mail.app itself to get corrupted.
(If you are the tinkering type, I assume it's possible to make modifications to the system volume from macOS Recovery, but that ability is made as inconvenient as possible—i.e. you have to restart your computer into a special maintenance mode, macOS Recovery, in order to do so—so that people won't casually bork their macOS or its core applications.)