[Updated, finished]: Disappointing real world results? Final Cut exports poor

[AppleFan22]

Has anyone else experienced really disappointing performance from their M1s compared to their Intels?

I bought the base MBA to try and replace my 13" 2018 MBP (i5 4x2.3 GHz with 8 GB RAM, 256 GB storage), since all the specs seemed so much better, and the keynote blew me away.

Then I started running benchmarks and render tests, and I was even more blown away. Results below for Blackmagic disk speeds, Geekbench results, and Cinebench results.

Obviously, I know that these are only benchmarks, and don't often reflect the realities of the device's capabilities, so I created a FCPX timeline with some 4K .mp4 files I had from some previous work. I cleared all the FCPX stores, turned off background rendering, and started the test.

The Intel beat the M1, noticeably. Took M1 20:04 minutes to export a 23:41 4K film, whereas the Intel did it in 14:30.

I can't tell why? The MBA is clearly the more powerful machine, surely? 8 total cores, running around 3.2 GHz? Plus 7 graphics cores, better disk speeds, and benefits of Apple Silicon?

I can't imagine heat throttling was an issue. MBP got hot but the fans stayed quiet enough I couldn't hear them, and the MBA stayed cold the entire time. Has anyone got any ideas why this might be?

Apologies the results aren't so pretty, I couldn't be bothered to make them look nice!

 

[AppleFan22]

Just repeated a similar test. I took a custom generator (thing that makes blobs appear on screen) and made it 5 minutes long. Then added another one, the same time, but more saturated. A total of a 10 minute video, at 4K.

M1 did it in 7:06, Intel did it in 21:55.

But now I wonder what's different?

Could be the power supply affecting performance? M1 did badly when I ran battery only tests, this time I did M1 on 61W charger, and Intel on battery alone.

 

[jeanlain]

The power supply should not affect performance.
Maybe you had spotlight indexing in the background or any other task running while you were doing your first 4k export.

Note that the blackmagic benchmark tools have not been ported to the M1, AFAIK, so their results may not be representative.

 

[Makosuke]

Since you're testing with an Air under heavy workloads, are you sure the difference wasn't thermal throttling?

I don't have any hard numbers at all, but you can assume that at normal room temperatures the Air can manage X seconds of operation at 100% CPU/GPU load before throttling, at which point it can only do Y seconds of 100% load per Z seconds of overall operation. If you ran a bunch of benchmarks that took roughly X seconds, then ran your FCPX test, it would now be in the thermally throttled "range", and would run much slower for a while.

I don't know what X is, but scattered game-benchmarking reports seem to be implying it's on the order of magnitude of 10 minutes.

If after your first poor result you then goofed around, did stuff, then ran a new test, it had time to cool down, and now you're back in the un-throttled range as long as your test took somewhere in the range of X seconds to complete.

It sounds like you didn't run exactly the same test both times, so it's also possible that the export you ran the first time happens to be something the i5 is well optimized for and the M1 isn't. To find out you could let it sit for a while, make sure it's cool, then re-run your first test and see if you get significantly different results.

For what it's worth, BareFeats did an FCPX export test (BruceX to ProRes 4444 XQ, which my understanding is uses generators, titles, and transitions to create a short 5K timeline then export it, testing a variety of FCP actions). Their result compared to the 2020 i7 13", which is modestly faster than yours, was 9s vs 48s, which is quite close to the difference you came up with in your second test.

 

[jeanlain]

Since you're testing with an Air under heavy workloads, are you sure the difference wasn't thermal throttling?

There's no way throttling can make an M1 slower than a core i5 from two years ago.

 

[PeterTheGreat17]

Just repeated a similar test. I took a custom generator (thing that makes blobs appear on screen) and made it 5 minutes long. Then added another one, the same time, but more saturated. A total of a 10 minute video, at 4K.

M1 did it in 7:06, Intel did it in 21:55.

But now I wonder what's different?

Could be the power supply affecting performance? M1 did badly when I ran battery only tests, this time I did M1 on 61W charger, and Intel on battery alone.

These are definitely interesting results. Can you run more tests?
I think Apple boasted about M1 chip’s capability in rendering subtitles and effects? That kinda explain the second run. Usually performance doesn’t drop too much on battery compared to plugged-in on MacBooks.

 

[AppleFan22]

Since you're testing with an Air under heavy workloads, are you sure the difference wasn't thermal throttling?

I don't have any hard numbers at all, but you can assume that at normal room temperatures the Air can manage X seconds of operation at 100% CPU/GPU load before throttling, at which point it can only do Y seconds of 100% load per Z seconds of overall operation. If you ran a bunch of benchmarks that took roughly X seconds, then ran your FCPX test, it would now be in the thermally throttled "range", and would run much slower for a while.

I don't know what X is, but scattered game-benchmarking reports seem to be implying it's on the order of magnitude of 10 minutes.

If after your first poor result you then goofed around, did stuff, then ran a new test, it had time to cool down, and now you're back in the un-throttled range as long as your test took somewhere in the range of X seconds to complete.

It sounds like you didn't run exactly the same test both times, so it's also possible that the export you ran the first time happens to be something the i5 is well optimized for and the M1 isn't. To find out you could let it sit for a while, make sure it's cool, then re-run your first test and see if you get significantly different results.

For what it's worth, BareFeats did an FCPX export test (BruceX to ProRes 4444 XQ, which my understanding is uses generators, titles, and transitions to create a short 5K timeline then export it, testing a variety of FCP actions). Their result compared to the 2020 i7 13", which is modestly faster than yours, was 9s vs 48s, which is quite close to the difference you came up with in your second test.

Ya, definitely not thermal throttling. Like I said, MBA was cold (literally, COLD to the touch) the whole time.

Not doing the same test as there seems to be something up with the camera footage. The first times I ran the test (not mentioned here) some of the footage was corrupted, which I didn't know about, which caused both machines to fail pretty fast. As well as this, it doesn't come up with the render icon in FCPX (if you don't know FCPX, there's a line that appears above a clip to indicate it needs to be rendered to ensure smooth playback). I've prevented FCPX from rendering until I get to export, so I don't know how these videos can be pre-rendered on BOTH machines (after deleting FCPX render files several times - that said, the delete render files options are different now so maybe I'm doing something wrong).

I've just done another test, exporting a video I already exported a few months ago. This one really bamboozled me. M1 FAILED the export JUST as it was doing the last bit. I forgot to record the reason, but it said something about Motion. The MBP did just fine, but took around 6 minutes longer.

I'm really not sure what's going on right now. I'm going to give this all up for now, and go to bed (it's 6am here!). I'll give it another go tomorrow with some fresh video (will record some 4K with iPhone XS and see what happens).

 

[AppleFan22]

The power supply should not affect performance.
Maybe you had spotlight indexing in the background or any other task running while you were doing your first 4k export.

Note that the blackmagic benchmark tools have not been ported to the M1, AFAIK, so their results may not be representative.

Possibly... I did turn off both Macs and let them sit for a while before doing anything else.

The power supplies I have are the 61 W charger which came with MBP and 30 W charger which came with MBA. Definitely wouldn't affect performance? My thoughts are that if it's plugged in and getting more wattage, it should, in theory, become more powerful (more power in = more power available = more power used?)? If this is the case, it doesn't really bother me, as I usually do the intensive stuff plugged in anyway.

If power supply WAS a contributing factor, there should have been better MBA performance when I ran tests on battery only... (MBA failed these too).

Ugh, this is too confusing. Might just keep the MBA anyway, it's prettier (and I've missed function keys SO MUCH).

 

[AppleFan22]

These are definitely interesting results. Can you run more tests?
I think Apple boasted about M1 chip’s capability in rendering subtitles and effects? That kinda explain the second run. Usually performance doesn’t drop too much on battery compared to plugged-in on MacBook

I will do more tomorrow, I just need to be somewhat creative in what kinds of tests I can do (don’t want to make it too hard to replicate on both machines, although I could maybe clone the FCPX library...)

 

[moonwalk]

Supposedly the M1 Air begins throttling about 9.5 minutes into an intensive task. So some slowdown can be expected. Is FCP native to M1 yet, or just universal?

 

[AppleFan22]

Supposedly the M1 Air begins throttling about 9.5 minutes into an intensive task. So some slowdown can be expected. Is FCP native to M1 yet, or just universal?

Native I believe, all Apple apps were supported from launch day (don’t quote me on this)

 

[Makosuke]

The power supplies I have are the 61 W charger which came with MBP and 30 W charger which came with MBA. Definitely wouldn't affect performance? My thoughts are that if it's plugged in and getting more wattage, it should, in theory, become more powerful (more power in = more power available = more power used?)? If this is the case, it doesn't really bother me, as I usually do the intensive stuff plugged in anyway.

If power supply WAS a contributing factor, there should have been better MBA performance when I ran tests on battery only... (MBA failed these too).

This isn't how Apple devices work when plugged in and power-limited.

So far as I know, everything Apple sells is capable of running at full power off of the battery.

If you plug a laptop (or an iPad, according to what I've seen) into a charger that can't supply enough power to satisfy what the computer is doing, it doesn't slow down, it just starts discharging the battery to make up the difference. If I plug my 15" MBP into a 20W charger instead of a 100W one, it doesn't suddenly slow down, it just says "not charging".

Even if the Mac really was power-limited by the battery, it wouldn't speed up when unplugged, it would at absolute worst run at the exact same speed, and speed up slightly if plugged into a higher-power charger that could supplement the battery.

If there were any doubts, just plug your MBA into your MBP charger to prove it. The 61W charger can obviously supply whatever the MBA needs, and plenty more than the 30W charger that came with the Air were the Air to request that much power.

 

[moonwalk]

Native I believe, all Apple apps were supported from launch day (don’t quote me on this)

Looks like it’s binary, not native:

New in Final Cut Pro 10.5​

Released November 12, 2020

• Improved performance and efficiency on Mac computers with Apple silicon.
  https://support.apple.com/en-us/HT201237

 

[Jouls]

Universal binary = native for Intel + native for M1

 

[AppleFan22]

Currently running what I'm going to consider the final and deciding test, assuming neither computer produces an error. Read on if you want to know what test I'm running.

I sat and recorded ~44 minutes of TV using a 4K camera (no RAW footage, just Canon mp4 files) pointed at the screen, as this would give the video files more heft. I've plopped those into a FCPX Library on an external SD card (the one the videos were recorded onto). Mainly for storage related issues, but I've also realised that it will even out the write speeds, so it is literally just the export encoding that will make a difference here. I also added a transition between two clips (Canon's EU recording limit meant I had to take two videos), and a generator to either end of the total clip.

As well as this, I've turned off bluetooth and wifi on the mac that's operating, so there won't be any iCloud upload/download interference (as there always is on my computers). Both in the same room, both plugged in and charging, sat on the same make of table. Not much more I can do to even it out!

I also realised why neither Mac wanted to render the video in my first test - it didn't have to. The video was completely unedited in FCPX, it was just placed side by side with each other (which isn't something I've ever done, hence why I didn't recognise FCPX wouldn't render the videos). To remedy this, I've adjusted the saturation on both clips, one to 0% and the other to 200%. This meant the whole timeline needs rendering.

MBP is running the export now, and has been going for over an hour, and is currently (time of writing) at 81%. I have an internal temp monitor open and CPUs have peaked at 95ºC (that I've witnessed), and, weirdly, the fans haven't really kicked on (max I've seen was around 2,000 RPM). Wonder why it's so tolerant to high temps?

Anyway, the test is still ongoing. Will report back once it's done.

 

[AppleFan22]

FINAL UPDATE:

Well, that was... interesting. It's an easy winner; the M1 MBA finished the export in 1:13, whereas the MBP finished it in 1:30. So whilst the MBA is better, it's not better by much.

Throughout the test, I tried estimating when the MBA would finish. It gained % points so fast it seemed like it could be finished in around 30 minutes. Then I estimated 40... then 50... then 60... then I gave up. For shorter projects, the MBA would seemingly have annihilated the MBP, which gained its % points much more steadily.

It's interesting to note that, during the MBA test, I opened Macs Fan Control to see what the temperature was of the SSD. It was reporting around 37ºC at the peak, and this seemed to be where the MBA's performance hit the temperature throttle plateau.

The MBA is the clear winner in all round short term speed though. The MBP took ages to even open FCPX, but the MBA opened it almost instantly.

This M1 road is going to be an interesting one, that's for sure. But right now, the M1 reigns supreme.

(side note: I would get the MBP, however, the MBA is prettier, and I can also make more money by selling the MBP than I paid for the MBA ?)

 

[MK500]

So, for this test, were you reading and writing to an SD card? I got a little confused about that part.

Obviously one of the big advantages of the M1 is improved IO speed. So while I respect the interest in only testing CPU/GPU performance, I think it's not a great real-world test.

Sorry if I'm confused about how you did the test. Maybe you were reading from SD and writing to the built-in SSD?

I'd recommend running all tests with the SD card out of the picture (copy the files to the internal SSD before beginning the project).

 

[Makosuke]

Well, that was... interesting. It's an easy winner; the M1 MBA finished the export in 1:13, whereas the MBP finished it in 1:30. So whilst the MBA is better, it's not better by much.

Throughout the test, I tried estimating when the MBA would finish. It gained % points so fast it seemed like it could be finished in around 30 minutes. Then I estimated 40... then 50... then 60... then I gave up. For shorter projects, the MBA would seemingly have annihilated the MBP, which gained its % points much more steadily.

Results report appreciated.

Sounds like pretty much the exact results you'd expect--for the first roughly 10 minutes the MBA was on track to be roughly three times as fast as the MBP, right up until it hit the thermal throttling point then had to slow way down. The estimate kept ratcheting up since the app didn't know why things had slowed down, only that its average progress was getting progressively worse after the step-change in speed.

Overall, impressive results still--even when it ran up against the thermal wall, a passively-cooled lower-end laptop with double the battery life still outpaced an actively cooled, higher-end one with an Intel chip by a decent margin.

It's extrapolating a lot, but if you trust that 30 minute estimate and assume that the thermal throttling kicks in after 10 minutes, you can guesstimate that the M1 is maybe 3x faster when it's not throttled, and when thermally limited it's right around the same speed as your i5 4x2.3. (My math is that if the estimate was 30 minutes it was initially progressing at a rate of 2-projects-per-hour, so finished about ⅓ of the work in the first 10 minutes, then took about an hour to do the remaining ⅔, while the MBP at constant speed was chugging along at ⅔ of the project per hour.)

 

[AppleFan22]

So, for this test, were you reading and writing to an SD card? I got a little confused about that part.

Obviously one of the big advantages of the M1 is improved IO speed. So while I respect the interest in only testing CPU/GPU performance, I think it's not a great real-world test.

Sorry if I'm confused about how you did the test. Maybe you were reading from SD and writing to the built-in SSD?

I'd recommend running all tests with the SD card out of the picture (copy the files to the internal SSD before beginning the project).

Hey, no worries, I was tired so wasn’t very coherent ? I was reading and writing to the SD card, the FCPX library was also on the card, as were the original clips. I would have done it from the internal disk but since I’ve only got the 256 GB model I don’t have a huge lot of disk space (the 4K files were 26 and 12 GB!). I’m trying to shed more data to iCloud Drive instead but it’s taking a while so go through everything! My workflow on my MBP would be to read and write FCPX to an external SSD to prevent storage issue (since FCPX libraries are enormous). Would love a 512 GB SSD but it doesn’t seem like the best idea idk

 

[AppleFan22]

Results report appreciated.

Sounds like pretty much the exact results you'd expect--for the first roughly 10 minutes the MBA was on track to be roughly three times as fast as the MBP, right up until it hit the thermal throttling point then had to slow way down. The estimate kept ratcheting up since the app didn't know why things had slowed down, only that its average progress was getting progressively worse after the step-change in speed.

Overall, impressive results still--even when it ran up against the thermal wall, a passively-cooled lower-end laptop with double the battery life still outpaced an actively cooled, higher-end one with an Intel chip by a decent margin.

It's extrapolating a lot, but if you trust that 30 minute estimate and assume that the thermal throttling kicks in after 10 minutes, you can guesstimate that the M1 is maybe 3x faster when it's not throttled, and when thermally limited it's right around the same speed as your i5 4x2.3. (My math is that if the estimate was 30 minutes it was initially progressing at a rate of 2-projects-per-hour, so finished about ⅓ of the work in the first 10 minutes, then took about an hour to do the remaining ⅔, while the MBP at constant speed was chugging along at ⅔ of the project per hour.)

Yeah you’re about right I think. It is a shame about the thermal throttling, given it really didn’t get very hot at all, but I suppose it all builds up in there over a while.

shame the M1 MBP works out more expensive, as I make money back by selling MBP!

 

[timshundo]

Might be a codec thing. I dont know what your Canon files are encoded as, "mp4" doesn't tell us much, but I think most of the jaw-dropping performance gains come from rendering 4k/6k/8k/ProRes/HDR/10-bit video which most high-end editors are working with. Have you tried creating "optimized media" for the video you're working with?

 

[timshundo]

Example here: https://alkhaleejtoday.co/internati...k-Pro-does-better-than-iMac-Pro-in-Final.html

 

[AppleFan22]

Might be a codec thing. I dont know what your Canon files are encoded as, "mp4" doesn't tell us much, but I think most of the jaw-dropping performance gains come from rendering 4k/6k/8k/ProRes/HDR/10-bit video which most high-end editors are working with. Have you tried creating "optimized media" for the video you're working with?

I believe they're H.264, how would I check that? Film isn't my (current) main job but I'm spending more and more time with a camera so it would probably be good to learn all this haha!

No, not done any optimised media. Never really thought to, to be honest. Something worth checking out?