Across the board or with AMD/RX580 specifically?
April 2008 Press Release:
https://www.nvidia.com/object/io_1208339777927.html
"Compatible with
any Intel-based Mac Pro, the GeForce 8800 GT includes NVIDIA’s video processing technology which
offloads H.264 decode enabling silky smooth playback of high-definition video."
H264
decode acceleration is (at least
supposed to be) present with NVIDIA GPUs dating back to 8800GT. This does not help anything with macOS Mojave, however. This report does nothing for HEVC or H264 encode. Do not expect full HEVC acceleration for encode on MP5,1 due to CPU. H264 is another story entirely and
should be compatible.
VideoProc is what many on this forum are citing for testing this, but there is no confirmation the software can/would properly enable or if MP5,1 is actually compatible in any capacity (aside from CPU-based). The manufacturer does not respond to any inquiries.
Sorenson Squeeze was compatible with H264 acceleration for
encode at one point, but the quality was garbage. Changes in Sierra (and later versions of El Cap) may have broke this since the older version of Squeeze was no longer supported on Mac. This MIGHT have something to do with OS changes due to Metal's first implementations, but I'm not 100% positive.