M3 vs M4

[Confused-User]

Of course there's also the ray tracing scores from M2 and M3, definitely another outlier of improvement for M3 - that could be down to core changes

Well of course. Did you miss that HW RT is one of the huge changes in the GPU in the M3 gen? (Or am I missing that that test isn't taking advantage of the GPU? Probably, but I'm too lazy to check 🙂. If so, yeah, that's interesting too.)

Anyway, good work. Very interesting!

 

[crazy dave]

Well of course. Did you miss that HW RT is one of the huge changes in the GPU in the M3 gen? (Or am I missing that that test isn't taking advantage of the GPU? Probably, but I'm too lazy to check 🙂. If so, yeah, that's interesting too.)

Indeed. This is CPU only 🙂

Anyway, good work. Very interesting!

Thanks!

 

[name99]

This is a very interesting table!

I'm not surprised that the IPC seems to be roughly on-par between the M1 and the M2 (we already knew that the majority of the performance gains came from clock speed increases), but I'm still very curious about the obvious outlier of the HTML5 scores. That's a 20% bump for a chip that otherwise has very similar IPC to its predecessor (and this isn't just an isolated test result, various other HTML5 benchmarks have shown similar gains, including Antutu's HTML5 tests and Speedometer 2.1).

Similar gains could be seen in object detection too, though I know less about that one in terms of other benchmarks that could verify the increases (not that we have any reason to doubt Geekbench, but I think it's good to verify with other benchmarks as well just to make sure it isn't a difference in how something was compiled). Does anyone know what technical changes Apple might have made within the chip to contribute to higher IPC that was seen on these benchmarks?

Love the data that you've shared here, definitely going to bookmark this post and probably refer to it.

Do these "HTML" tests also test Javascript?
One of the changes between M1 and M2 was an improvement to branch prediction that improves a very particular sort of use case where something like an interpreter jumps through an indirect branch to one of multiple end points. The change was made to boost Objective-C method performance, but also helps interpreters of a particular sort. But code has to make one particular small change to make use of this new functionality.

I can't remember if we actually ever validated it, but it's an obvious assumption that Apple's JS interpreter was tweaked to do this. Other interpreters (PDF? and third party things like Lua?) might also be able to use it, IF they new about, and bothered to make, the change.

Point is, this may be the source of the big "HTML" jump for M2.

 

[CWallace]

My bet is Studio and maybe Pro M4 desktops at WWDC.

If Apple has the M4 Max and Ultra ready, I agree. If they also have the M4 Pro ready, I could see Mac mini being updated, as well.

Pro laptops at WWDC or July (yes July would be atypical).

I do not believe we will see MacBook Pro updates before October / November 2024.

Air July or October/November.

Not happening before March 2025, IMO.

 

[leman]

Here is a slightly refined version of the analysis @crazy dave did. I took multiple GB6 results, normalized them by frequency, resampled the values, and plotted the resulting ratios. Here, 1 means same iso-clock performance, >1 means M4 is faster, <1 means slower. For tests where M4 is consistently faster than M3 the bulk of the distribution will be over 1.

 

[Confused-User]

Here is a slightly refined version of the analysis @crazy dave did. I took multiple GB6 results, normalized them by frequency, resampled the values, and plotted the resulting ratios. Here, 1 means same iso-clock performance, >1 means M4 is faster, <1 means slower. For tests where M4 is consistently faster than M3 the bulk of the distribution will be over 1.

View attachment 2376939

I noticed this the other place you posted it (I sometimes lurk there, though really I should just move there, way better S/N).

Great work, but if possible for future efforts, I'd suggest shade-banding on the Y axis. That would make it much easier to follow as your eye moves right along the X axis. Even two alternating backgrounds every .15 or .2, say, would be great. If that's not feasible with your software, maybe just dark grid lines every so often.

 

[MrGunny94]

I do think we should wait for a Mac model when discussing the M4. The iPad is a great device but honestly there’s a lot of things missing from the base iPad chip and the OS to back it up.

Really waiting to see what Apple does with the Mac lineup come the M4

 

[leman]

Great work, but if possible for future efforts, I'd suggest shade-banding on the Y axis. That would make it much easier to follow as your eye moves right along the X axis. Even two alternating backgrounds every .15 or .2, say, would be great. If that's not feasible with your software, maybe just dark grid lines every so often.

That’s a great suggestion. I didn’t have much time to tweak the plot, hence default settings. I agree I should have at least highlighted some of the ticks.

 

[jeanlain]

A line at y = 1 would be easy to add. But how do you compute M4 minus M3 performance given that data aren't paired?

 

[Dulcimer]

Should we expect more improvements to CPU and especially GPU uarch in the fall when M4 Pro and M4 Max launch in half a year (current rumors)?

I would think not since they’re just supposed to scale up from the same cores in M4. But I’m also hoping that if MBPs won’t be updated for another half year that some more improvements would be added.

 

[MrGunny94]

Should we expect more improvements to CPU and especially GPU uarch in the fall when M4 Pro and M4 Max launch in half a year (current rumors)?

I would think not since they’re just supposed to scale up from the same cores in M4. But I’m also hoping that if MBPs won’t be updated for another half year that some more improvements would be added.

Most likely yes and I bet the Pro will have some good battery life with that 6 E-Core cluster setup.

We really need to wait and see the battery of life of the M4 in the Mac first to draw some conclusion for the Pro lineUp though

 

[leman]

A line at y = 1 would be easy to add.

I’ll try to improve the plot tomorrow morning.

But how do you compute M4 minus M3 performance given that data aren't paired?

Resampling. I pick 10000 data pairs at random with replacement. And I calculate ratios, not differences.

 

[Confused-User]

Should we expect more improvements to CPU and especially GPU uarch in the fall when M4 Pro and M4 Max launch in half a year (current rumors)?

I would think not since they’re just supposed to scale up from the same cores in M4. But I’m also hoping that if MBPs won’t be updated for another half year that some more improvements would be added.

Nobody has a clue, because Apple has definitively broken almost all its patterns and habits this year, not that it had really established much of a pattern to begin with since the advent of the M1.

I think three generations in 12-13 months is unlikely, @leman thinks it's likely, but neither of us has anything stronger than general industry knowledge to base those feelings on. Neither does anyone else outside Apple and TSMC.

I expect that we'll see those M4s much sooner than that - at WWDC, most likely - and that they'll be all the same function blocks as the M4, but possibly running at somewhat higher clocks (4.5-4.8GHz in the Studio, lower in other devices). What will be really interesting is if they're ready to show more of their hand on the high end - say, a SoC for the Studio or Mac Pro with more than just two chips back-to-back. Regardless I would expect further improvements in scaling for the larger chips. And there's a pretty decent chance we'll see bigger memory bandwidth improvements on the desktops than shown by the M4 on the iPad. So, instead of 480GBps for the Max, we'll likely see 533GBps, though slower and faster are possible (Micron's shipping LPDDR5x at 9.6GT/s, which would push the Max to 600GBPs).

 

[DrWojtek]

It will be funny to hear what Gurman has to say this Sunday. He must have realized by now that there is no way not a single mac gets M4 until end of 2024, as was his prediction before the M4 release.

My guess: desktops released with M4 this WWDC. Including Mac Pro and therefore Ultra. Might not be ready to buy yet, but at least showcased. Excluding iMac.

If Apple wants to release MBPs with M4’s, they’ll come now, too. If not, they’ll make an announcement of that. Something like ”the M4 series is now complete”. As not to hold people off from buying the M3 MBPs in anticipation of M4 release.
The Macbook Air will not get M4.

If no M4 MBPs now, we will probably see M5 MBPs as soon as whatever they are waiting for is finished. Probably early next year.

If we get M4 MBPs now I believe we will see M5 next WWDC for the entire line and it will have 12 GB of RAM as base, a great GPU boost and little bit more marketed on gaming than usual (and AI, of course).

 

[Confused-User]

Ok, one more thought on this and then (I hope) I'm done.

They could move *everything* to M4 at WWDC. For the Airs (or maybe all laptops?), they could sell the M4s at a $100 price premium over the M3s, while continuing to sell the M3s. That way there's less of a sting for recent buyers of the M3. Then maybe in the late summer or fall they eliminate the M3s and move the prices down for the M4s.

I don't think this is actually going to happen, but it is a plausible way of moving more units to M4 earlier while mitigating the inevitable backlash from M3 buyers unhappy with their fairly-new M3s being dethroned.

 

[DrWojtek]

Ok, one more thought on this and then (I hope) I'm done.

They could move *everything* to M4 at WWDC. For the Airs (or maybe all laptops?), they could sell the M4s at a $100 price premium over the M3s, while continuing to sell the M3s. That way there's less of a sting for recent buyers of the M3. Then maybe in the late summer or fall they eliminate the M3s and move the prices down for the M4s.

I don't think this is actually going to happen, but it is a plausible way of moving more units to M4 earlier while mitigating the inevitable backlash from M3 buyers unhappy with their fairly-new M3s being dethroned.

I thought about that as well. I think there is also the question of, How much stock of M3 Pros/Max do we have left? Chips and complete units.

If they are almost out of M3, they’ll jump on M4 as quick as possible. I don’t think they give a rats ass about what people who bought M3 would feel. At the end of the day Apple strive to be inspiring, not customer-oriented. Apple thinks they’ll win customers back anyways if they are inspiring.

On the other hand, if they’re sitting on loads of M3 they won’t release M4 MBPs at all. Few would buy an M3 if M4 is right there, even if it’s more expensive. They know we have the cash, and if we don’t we borrow it.

That decision has probably already been made, and what the answer is, we will probably see soon. The heavy discounts on M3 laptops might be a clue; they want to rid the stock and go full M4, asap.

 

[leman]

@Confused-User @jeanlain Improved the plot based on your suggestions, thanks again!

 

[Pressure]

I just want to point out that the Mac mini has gone 481 days without an upgrade and the Mac Pro as well as Mac Studio have gone 341 days.

The rest of the line-up has been upgraded to newer SoCs.

 

[jeanlain]

Resampling. I pick 10000 data pairs at random with replacement. And I calculate ratios, not differences.

Yes ratios of course.

I wonder how I would proceed if I wanted to estimate a ratio between independent variables. I don't think reviewers would like the idea of generating artificial pairs, but a bayesian estimate would be much more complex to obtain.

Nice plot overall. If I'm allowed to nitpick, I would put the line a y = 0 behind the violin plots.

 

[Torty]

I thought about that as well. I think there is also the question of, How much stock of M3 Pros/Max do we have left? Chips and complete units.

If they are almost out of M3, they’ll jump on M4 as quick as possible. I don’t think they give a rats ass about what people who bought M3 would feel. At the end of the day Apple strive to be inspiring, not customer-oriented. Apple thinks they’ll win customers back anyways if they are inspiring.

On the other hand, if they’re sitting on loads of M3 they won’t release M4 MBPs at all. Few would buy an M3 if M4 is right there, even if it’s more expensive. They know we have the cash, and if we don’t we borrow it.

That decision has probably already been made, and what the answer is, we will probably see soon. The heavy discounts on M3 laptops might be a clue; they want to rid the stock and go full M4, asap.

Don’t understand apples strategy. They could have released the air a bit later but with M4. They could have released a refreshed iPad Pro with M3 earlier and double dip with latest OLED model later.

 

[leman]

I wonder how I would proceed if I wanted to estimate a ratio between independent variables. I don't think reviewers would like the idea of generating artificial pairs, but a bayesian estimate would be much more complex to obtain.

What I am doing is essentially comparing the distributions, and resampling techniques like these are fairly commonplace in academia. So I don't think reviewers would mind that particular bit, after all, the observations are not correlated.

If you want to apply statistical testing to it the trick would be finding a suitable model. I suppose one would look at Cauchy distribution since it describes the ratio of normally distributed variables, but I am not knowledgeable in that particular domain and don't have an intuition.

For simple estimates however working with the bootstrapped empirical distribution like what I use works very well. E.g. you can easily evaluate how much of the probability mass is > 1 or < 1, which will give you an idea about a possible effect strength and its direction. This particular application of probability follows the spirit of Cox's theorem in the sense that we can quantify our certainty about a variable by exploring the contrasts in its behavior.

Nice plot overall. If I'm allowed to nitpick, I would put the line a y = 0 behind the violin plots.

Do you mean y = 1 (the no difference baseline)? I updated the plot this morning adding the baseline and a shaded background to make the plot more readable.

 

[leman]

Don’t understand apples strategy. They could have released the air a bit later but with M4.

And that is why I don't believe they plan to release Air with an M4 at all.

We also need to consider things we do not see. It is entirely possible that manufacture of M4 is constrained at this moment, so they cannot ship it in too many products.

 

[Retskrad]

Why are people believing the marketing that the M4 is magically a whole new powerhouse chip for AI tasks? A17 Pro NPU can do already do 35 TOPS (vs 38 TOPS in M4). What new ground is Apple breaking here that only the M4 is capable of?

 

[DrWojtek]

Don’t understand apples strategy. They could have released the air a bit later but with M4. They could have released a refreshed iPad Pro with M3 earlier and double dip with latest OLED model later.

Cant have Air with better SoC than Pro. Also use Air to get rid of M3 stock. M3 probably too hot to fit in iPad plus need new AI marketed chip. Plus wouldnt want to keep M3 in production for so long. M4 iPad Pro might have longer cycle than Air.

I think they originally had planned to release M3 for everything. Half way in they realize either 3nm B is worse than anticipated or 3nm E is better than anticipated. They decide to cut B short and only for highest moving products (MBPs) since they were constrained with contracts to TSMC.

Then it turns out things are moving slower than they had hoped for. They release it on more models (iMac and Air) to get rid of more M3 stock.

Combine this with the need to up the game on AI marketing and that they have to update the desktops… and we’ll get new desktops next month and new MBPs either at the same time or shortly after.

Air and iMac will not get M4.

 

[Carrotstick]

What new ground is Apple breaking here that only the M4 is capable of?

The M4 supports the SME extension.