Apple M1 VP9/AV1 decoding?

[galad]

Ventura ships both a software decoder and an hardware decoder VideoToolbox components.
But at the moment the front facing softwares are enabled only to decode avif images, AV1 in mp4 and mov containers is not enabled yet.
I didn't check whether the hardware decoder is used on Apple Silicon, and if so on which CPU family.

 

[Botts85]

AVIF is working in Safari, Finder, and Preview. I used the Netflix samples here to test: http://download.opencontent.netflix.com/?prefix=AV1/Chimera/AVIF/

Ventura ships both a software decoder and an hardware decoder VideoToolbox components.

That means it should work in QuickTime though. Right now QuickTime is a no go.

Safari 16.1 and QuickTime 10.5 won't play back Netflix's AV1 Chimera sample on an M1 with Ventura. VLC will play the 1080p version of Chimera, but at 100% CPU usage.

I had the same results with all test files from http://av1.webmfiles.org. VLC plays them back, but in software. Quicktime, Quick Look, and Safari won't.

iOS 16.1 also will not play back AV1 in .mp4 containers.

I'm somewhat relieved that M1 / M1 Pro actually handles VP9 and AV1 playback with minimal heat/fans/power draw despite not having true hardware decode. Not sure how they manage it! Was super hesitant and pissed when they came out and didn't announce that support.

VP9 isn't bad for battery use, but trying out some AV1 1080p content today, I was seeing 77-100% CPU usage (with my activity monitor running 0%-800%) in VLC.

 

[galad]

No it doesn't mean it should work in QuickTime. Some components are disabled and not loaded by default.

 

[EugW]

Since M2 got a significant improvement in the media engine over M1, I could see Apple turning on AV1 acceleration in hardware for M2 without including M1. OTOH, even if it’s already there in both M1 and M2, there’s no guarantee Apple would activate it for either of them.

I still remember iPhone 6 with A8 got HEVC encoding for FaceTime but then Apple just stopped talking about it completely. Ultimately later when Apple went full tilt with HEVC, Apple left out A8.

Anyhow, I’ll be buying an M2 Mac mini (or maybe an M2 Pro Mac mini if released). We will see if it ever gets official AV1 support.

 

[Piipperi]

Since M2 got a significant improvement in the media engine over M1, I could see Apple turning on AV1 acceleration in hardware for M2 without including M1. OTOH, even if it’s already there in both M1 and M2, there’s no guarantee Apple would activate it for either of them.

I still remember iPhone 6 with A8 got HEVC encoding for FaceTime but then Apple just stopped talking about it completely. Ultimately later when Apple went full tilt with HEVC, Apple left out A8.

Anyhow, I’ll be buying an M2 Mac mini (or maybe an M2 Pro Mac mini if released). We will see if it ever gets official AV1 support.

It's not about official support. If there's no HW accelerator, it'll just use SW for AV1. That's likely how FaceTime worked as well on the iPhone 6, because if you encoded high bitrate 1080p60 footage on a software encoder with the performance of the A8, your battery life would be really poor and it may not be even able to encode it in real time. FaceTime is low bitrate and low bitrates take nothing to really encode.

 

[EugW]

It's not about official support. If there's no HW accelerator, it'll just use SW for AV1. That's likely how FaceTime worked as well on the iPhone 6, because if you encoded high bitrate 1080p60 footage on a software encoder with the performance of the A8, your battery life would be really poor and it may not be even able to encode it in real time. FaceTime is low bitrate and low bitrates take nothing to really encode.

If there is no official support, there would be no hardware support at all because Apple doesn't expose the functionality to 3rd party applications.

In terms of FaceTime, even at those bitrates the iPhone would have battery life issues with software encoding via HEVC, and the iPhone would get hot, but it didn't have either of those issues. I'm guessing that likely the hardware was there, but only for reduced resolutions and only exposed for FaceTime. There was no need to use HEVC in software, because hardware h.264 was already supported and the most commonly used format, since everything before iPhone 6 was h.264.

I suspect the iPhone 6 A8 was excluded later from official HEVC support because its (presumed) hardware support was too low a resolution, only appropriate for FaceTime. Official HEVC support starting with A9 specified 4K.

 

[nquinn]

What's most perplexing to me is Apple hiding any information on if these chips support AV1 or not! (same for VP9 I suppose).

 

[MrGunny94]

What's most perplexing to me is Apple hiding any information on if these chips support AV1 or not! (same for VP9 I suppose).

Yeah it doesn't make any sense at all, but at the end of the day it is what it is, it's only a bunch of us discussing this on a forum.

 

[Piipperi]

What's most perplexing to me is Apple hiding any information on if these chips support AV1 or not! (same for VP9 I suppose).

Apple never exactly hid it, since Big Sur added VP9 decoding support for intel chips before M1 came out. For some reason Apple just really dislikes open standards like Opus, VP9, AV1, etc. They're only using them as needed when they really feel pressurised to support them (which is why you can't play VP9 videos outside of Safari without the use of 3rd party apps, and they had to add VP9 support so people wouldn't switch to Chrome to play 4K YT)

 

[Pressure]

Apple never exactly hid it, since Big Sur added VP9 decoding support for intel chips before M1 came out. For some reason Apple just really dislikes open standards like Opus, VP9, AV1, etc. They're only using them as needed when they really feel pressurised to support them (which is why you can't play VP9 videos outside of Safari without the use of 3rd party apps, and they had to add VP9 support so people wouldn't switch to Chrome to play 4K YT)

Hate might be the wrong word.

They are after all part of the AOMedia Steering Committee driving AV1 and do support VP9.

 

[nquinn]

Hate might be the wrong word.

They are after all part of the AOMedia Steering Committee driving AV1 and do support VP9.

True, though they were like the LAST big company to join. They have rights to HEVC and love making money from that licensing.

 

[Piipperi]

Hate might be the wrong word.

They are after all part of the AOMedia Steering Committee driving AV1 and do support VP9.

But if Apple didn't hate them why are those unsupported outside of Safari/WebKit applications?

 

[Piipperi]

True, though they were like the LAST big company to join. They have rights to HEVC and love making money from that licensing.

yeah Apple was practically forced to do it. And AV1 is pushed by Netflix and YouTube so they'll need to add the support one day either way. They were probably hoping with VP9 that it wouldn't have caught on that much which is why it took them literally nearly 4 years to add the support for VP9 HW decoding on Intel. Also just to add to that whole HW thing, I sent a Feedback asking them about VP9 HW decoding on AMD hardware, and I got a response that it is not supported and have no plans to do so. So they want to do the minimal effort to support such codecs really.

 

[Botts85]

They have rights to HEVC and love making money from that licensing.

Do we know that? I thought it was mostly a wash:

Provide patents to pool -> don’t pay a per Mac/iOS device license.