M4+ Chip Generation - Speculation Megathread [MERGED]

[tenthousandthings]

I think you make some very good points, but I want to take issue with a few specific things quoted here.

TSMC's roadmap isn't set in stone. It never is because they aren't the final arbiter, physics and engineering are, and it's not done until it's done. They have a fairly good record, but they have famously screwed up once recently (N3B rollout delay), and revised a released roadmap recently (moving BSPD down the road a bit). None of this really impacts your main point, though.

I don't see why you think volume production for the M4Pro started late *last year*. That seems wildly unlikely. I think if they were allocating wafers to the M4Pro they'd release it (along with the Max) earlier. More likely they went with the base M4 only to reduce the impact of defects as they ramp. Unless you just mean that N3E started volume production last year, not the M4Pro?

Your "fork in the road" prediction is interesting but I think you may be reading too much into the TSMC comments you quoted. Advanced packaging may reduce area substantially while raising z-height a bit. I think they can afford some z if it buys them area, but there's always heat dissipation to consider. So it's not at all clear to me how that's going to play out.

On "set in stone" -- I meant it almost literally, because it involves construction projects. On the roadmap, I just meant that by the time we hear about it, whatever adjustments they've had to make are in the past. Part of that comes from trying to look back and find (to refute Crazy Dave) an early TSMC statement where they said N3 and N3P, before it changed to N3"B", N3E, and N3P. But I couldn't find it. Nothing specific. They generally stay vague early on, without real specifics. It gets more detail as things get built and they have more confidence...

I think you are wrong about N3 -- the first date given (in 2020) for that was H2 2022, which they hit. The 2019 comments that preceded it were not specific. That's my point, by the time TSMC provides a specific date, things are pretty much set.

You are right about BSPD, but I don't remember them commenting on it other than in general terms (like, "yes, we are working on that") until they announced Super Power Rail. But I could be wrong. I haven't looked, and I seem to recall something about N2P getting BSPD, but it seems possible they just changed the name of the node to A16 at the same time Marketing came up with "Super Power Rail." I notice "N2P" has disappeared from the official discussion of N2...

On M4 Pro, I didn't mean to imply volume production started right off the bat along with M4 and presumably A18.

Oh, I'm absolutely certain I'm reading too much into those questions and comments... And my half-baked theory about a fork in the road due to both N2P and A16 coming online at the same time depends on that actually happening. If A16 has replaced N2P, then it’s not possible, obviously.

 

[Confused-User]

"So the M4 Pro was finalized in H2 2021 or H1 2022."
In H2 2021 or H1 2022 artificial intelligence to Apple meant Siri, Apple Car self driving or further perfecting the look of backlit woollen sweaters with the iPhone camera....

Surely that means any current talk about the M4 being an AI Great Leap Forward is mostly hype, with a stock split being done to double the neural engine benchmarks?

I am curious what you mean by this?

He's talking about supporting INT8 so they get double the TOPS compared to INT16 or FP16. Which IIRC came with the A17 first.

You could reasonably claim that that change is an inexpensive one-shot improvement, with no more gas in that particular tank, and you'd be right. (Well, except for the possibility of supporting INT4 or even smaller formats in the future, but in any case there's a clear limit to that sort of improvement.) But it's not just hype, supporting INT8 at double the speed is a real improvement.

Also, IIUC, while specific applications of AI hardware may have shifted, the desirability of various types of target hardware hasn't changed all that much in the last few years. That may not continue to be the case, though.

 

[leman]

He's talking about supporting INT8 so they get double the TOPS compared to INT16 or FP16. Which IIRC came with the A17 first.

Has this been definitively established? I was under impression that INT8 and FP16 rate of Apple NPU is identical (since they can use the same 10-bit multiplier hardware). D

 

[innerproduct]

I wonder, have you (people of this thread) come to any conclusions on why the “cadence” was lost and we got a m3max with RT last year but not an ultra. Why the m4 is parallel to the m3 and what the rumors for high end chips for studio and mac pro is and hand and plausible timeline? I really hope for a ultra (or better) desktop solution this fall but is that even in the cards?

 

[leman]

I wonder, have you (people of this thread) come to any conclusions on why the “cadence” was lost and we got a m3max with RT last year but not an ultra. Why the m4 is parallel to the m3 and what the rumors for high end chips for studio and mac pro is and hand and plausible timeline? I really hope for a ultra (or better) desktop solution this fall but is that even in the cards?

My guess that it has to do with manufacturing bottlenecks. These are new processes after all, maybe Apple wanted to make sure they can ship the popular products first. The M4 is a weird one since it was so close to the MBA. But this could also be driven by manufacturing constraint (N3E is a very new process after all). I think the desktop timeline is anyone's guess. Maybe we will get M3-based hardware, maybe M4-based, or maybe we will get something new entirely.

On that note, I originally though that we might get an M5 with some new hardware capabilities this year, not I am coolign down on this possibility. Big hardware innovations for Apple are usually preceded by patent publications. For example, RT tech patents were published in 2022, M3 GPU patents were 2022-2023, M3 CPU patents (branch predictor) were in 2023. But in the last few months there has been very little activity on the patent front, and it seems like most published patents were already implemented. There is a bunch of patents that describe ECC memory techniques on Apple Silicon, and also a large amount of advanced NPU-related patents. We don't know whether any of these have been implemented yet, so maybe this tech will make a debut this fall. Of course, all of this is fairly week evidence, I am only mentioning it because patents were a reliable indicator in the past.

 

[Confused-User]

Has this been definitively established? I was under impression that INT8 and FP16 rate of Apple NPU is identical (since they can use the same 10-bit multiplier hardware). D

I asked this question maybe 6-8 weeks ago, and I believe it was answered in the affirmative by a source I considered trustworthy at the time. But now I don't recall where that was - probably the other place. So I won't swear that this is correct, but I believe it to be: The A17 and M4 have INT8-capable NPUs, and INT8 is ~double performance of FP16.

 

[Confused-User]

I wonder, have you (people of this thread) come to any conclusions on why the “cadence” was lost and we got a m3max with RT last year but not an ultra. Why the m4 is parallel to the m3 and what the rumors for high end chips for studio and mac pro is and hand and plausible timeline? I really hope for a ultra (or better) desktop solution this fall but is that even in the cards?

I agree with leman's answer. Further, it's clear now that Apple never intended for there to be an M3 Ultra- at least one analogous to the M2 Ultra, two M2 Max chips "glued" together. Why, we don't know, but there are many plausible reasons. If I had to guess it would be some combination of process tech issues/choices (including the need not to steal too much wafer supply from the iPhone), and the impact of Apple's need for a huge number of AI-capable processors to support Apple Intelligence offloading. For all we know, they've been making M4 Ultras of some sort or another for months, and they're all going into Apple data centers.

 

[tenthousandthings]

I agree with leman's answer. Further, it's clear now that Apple never intended for there to be an M3 Ultra- at least one analogous to the M2 Ultra, two M2 Max chips "glued" together. Why, we don't know, but there are many plausible reasons. If I had to guess it would be some combination of process tech issues/choices (including the need not to steal too much wafer supply from the iPhone), and the impact of Apple's need for a huge number of AI-capable processors to support Apple Intelligence offloading. For all we know, they've been making M4 Ultras of some sort or another for months, and they're all going into Apple data centers.

Second Third this. I've stopped trying to see a cadence beyond extrapolating from existing patterns and TSMC's public roadmap. Thus the extrapolation I posted earlier doesn't include Pro/Max/Ultra releases beyond the M1 Pro/Max watershed:

M1 Pro/Max (2021) = N5 (2020)
M2 (2022) = N5P (2021)
M3 (2023) = N3 (H2 2022)
M4 (2024) = N3E (H2 2023)
M5 (2025) = N3P (H2 2024)
M6 (2026) = N2 (H2 2025)
M7 (2027) = A16 (H2 2026)

We don't know why the M3 Ultra didn't happen. Apple can be agile, and I don't see them settling into a set cadence with regard to the Ultra builds. They will probably skip a generation again, but when that would be is difficult to say. My guess is that it is all about the expensive and volatile world of advanced packaging, and we won't learn about concrete advances until (at best) a month or so before they appear in a shipping product.

We should be celebrating the fact that not just the Mn, but also the Mn Pro/Max series now appear to both have an annual cadence. This time last year I was arguing against this likelihood. Now it's real, as far as we know. M3 (and N3) changed everything. Maybe that's why there was no Ultra. One step at a time...

 

[JouniS]

We don't know why the M3 Ultra didn't happen.

The most likely reason is that Apple is a large organization. And like all large organizations, it's a dysfunctional bureaucracy. There is no grand plan. Only factions of middle managers fighting for status and power.

It's been a long time since Apple has released Macs on any consistent schedule. Macs are no longer a priority, and nobody spends too much effort to ensure that they are released on time. They will be released when they are ready, and when they fit in the marketing schedule. Or maybe they are not released at all, because someone with more status and power had other ideas, which are not compatible with the upgrade. Or maybe some key people get promoted/reassigned or leave the company, and the half-finished upgrade they were responsible for never sees the market.

 

[name99]

"So the M4 Pro was finalized in H2 2021 or H1 2022."
In H2 2021 or H1 2022 artificial intelligence to Apple meant Siri, Apple Car self driving or further perfecting the look of backlit woollen sweaters with the iPhone camera....

Surely that means any current talk about the M4 being an AI Great Leap Forward is mostly hype, with a stock split being done to double the neural engine benchmarks?

Please don't speak unless you actually like KNOW something about the subject...

Apple introduced the first "real" ANE in 2018 with the A12, AMX in 2019 with the A13.
They introduced BNNS and MPSCNN (neural net primitives on the CPU and GPU respectively) in 2016 and CoreML in 2017.
Every WWDC since 2017 has included talks about how to use the latest ML APIs, for vision (including non-obvious stuff like pose detection or finger/hand detection), motion classification, and various text tasks.

They just published a paper comparing the quality of Apple Foundation Models for text against the competition, and Apple comes off a whole lot better than you'd expect from a company that, by your theory, scrambled to put something together three months ago...

If you have no idea about all this background, well, that's on you mate.

 

[name99]

"If Apple was planning a self-driving vehicle then they would need a much more powerful NPU..."

Wasn't the planned Car release date 2025+?
That's my point.
Surely the M4's NPU was designed for reasons far removed from what today's AI hype is all about?
The AI Great Leap Forward will be more like 2026+?

Can you make your point using technical vocabulary and without snark?

You SEEM to be trying to say that the ANE is optimized for vision at the expense of text.
Do you have any evidence for this claim?
Definitely the ANE is optimized for vision; what's much more contentious is that this HURTS its text ability. Certainly the Apple Foundation Models for text numbers are very impressive.

It is probably the case that idiots who don't know what they are doing make a hash of trying to target an existing text model (eg a Llama variant) to the ANE. That tells us a whole lot about their incompetence, but very little about how the ANE can perform when given a text model designed to play to its strengths...

 

[name99]

He's talking about supporting INT8 so they get double the TOPS compared to INT16 or FP16. Which IIRC came with the A17 first.

You could reasonably claim that that change is an inexpensive one-shot improvement, with no more gas in that particular tank, and you'd be right. (Well, except for the possibility of supporting INT4 or even smaller formats in the future, but in any case there's a clear limit to that sort of improvement.) But it's not just hype, supporting INT8 at double the speed is a real improvement.

Also, IIUC, while specific applications of AI hardware may have shifted, the desirability of various types of target hardware hasn't changed all that much in the last few years. That may not continue to be the case, though.

ANE from day one supported both FP16 and INT8. This is not secret.

 

[name99]

My guess that it has to do with manufacturing bottlenecks. These are new processes after all, maybe Apple wanted to make sure they can ship the popular products first. The M4 is a weird one since it was so close to the MBA. But this could also be driven by manufacturing constraint (N3E is a very new process after all). I think the desktop timeline is anyone's guess. Maybe we will get M3-based hardware, maybe M4-based, or maybe we will get something new entirely.

On that note, I originally though that we might get an M5 with some new hardware capabilities this year, not I am coolign down on this possibility. Big hardware innovations for Apple are usually preceded by patent publications. For example, RT tech patents were published in 2022, M3 GPU patents were 2022-2023, M3 CPU patents (branch predictor) were in 2023. But in the last few months there has been very little activity on the patent front, and it seems like most published patents were already implemented. There is a bunch of patents that describe ECC memory techniques on Apple Silicon, and also a large amount of advanced NPU-related patents. We don't know whether any of these have been implemented yet, so maybe this tech will make a debut this fall. Of course, all of this is fairly week evidence, I am only mentioning it because patents were a reliable indicator in the past.

Remember that the patents only get *published* some time after submission :-(

When you look at the past you can see the pattern laid out. But when you try to imagine the future, problem is most of the patents that might be relevant to M4 family are still in processing. (Both being analyzed and I don't know if Apple can ask for "holds" to be put on them, to only be published after some "reasonable' future date, presumably after some relevant product launches.)

Another case you missed is the vector DSP patent. This MIGHT be modem related, but the names on it are mostly the names behind the original ANE patents. So? New functionality for ANE? Or they felt ANE was a solved problem and they'd tackle something new and different?

 

[Confused-User]

The most likely reason is that Apple is a large organization. And like all large organizations, it's a dysfunctional bureaucracy. There is no grand plan. Only factions of middle managers fighting for status and power.

It's been a long time since Apple has released Macs on any consistent schedule. Macs are no longer a priority, and nobody spends too much effort to ensure that they are released on time. They will be released when they are ready, and when they fit in the marketing schedule. Or maybe they are not released at all, because someone with more status and power had other ideas, which are not compatible with the upgrade. Or maybe some key people get promoted/reassigned or leave the company, and the half-finished upgrade they were responsible for never sees the market.

That's a very sweeping statement with no supporting data.

While Apple's made some missteps over the years, and you can argue that the car is a very big one (though I don't think that that's at all obvious), the disastrous picture you paint seems to be very far from the truth.

In fact, I know a few people who work at Apple. And I've heard some stories that aren't encouraging, about certain specific parts of Apple - though it seems clear to me that those parts of Apple were always going to be the messiest and most poorly run (I won't say more because I don't want to reveal any sources). But I've also seen some remarkable execution. And both of those data points are barely worth mentioning, as anecdotal "evidence" about an *enormous* organization.

What we do know is that as far as can be seen from the outside - which is a lot, though not everything - the part of Apple that's been delivering Apple Silicon has been firing on all cylinders for over a decade. It's one of the most remarkable stories in the history of processor design.

How this relates to Macs is not entirely clear. And it *is* clear that Macs are not as well managed as Apple Silicon is. (I am not a fan of how they're doing OS development, but I don't want to open that can of worms in this post.) But there just isn't enough evidence to say what Apple is thinking, yet. Too much has changed too rapidly in the last 4 years.

FWIW, I think the Mac is not Apple's top priority, but it *is* a priority because it has an important role to play in the future of their data centers, as well as being a significant contributor to revenue. Unlike some people I think PCC is a big step and one that they will continue to invest in for the foreseeable future, and I think it will depend on M chips, or a close variant that will share R&D costs with M chips. Macs will benefit strongly from that.

 

[Confused-User]

ANE from day one supported both FP16 and INT8. This is not secret.

Right, and I should have been clearer that what I was talking about was the new development that INT8 is doubly fast in the latest silicon. But as you can see, even in this thread, there's a lot of confusion about that.

 

[name99]

Right, and I should have been clearer that what I was talking about was the new development that INT8 is doubly fast in the latest silicon. But as you can see, even in this thread, there's a lot of confusion about that.

This is one of these claims that is endlessly repeated on the internet. But repetition doesn't make it true.
I keep waiting for the evidence of this supposed 2x INT8 throughput and I never see it. But of course the sorts of people who repeat this claim don't bother with pesky details like "evidence".

Look at

iPhone 15 Pro Max vs iPhone 14 Pro - Geekbench

browser.geekbench.com

It's not a perfect match because I couldn't be bothered to search for a perfect 1:1 comparison (in particular note the iOS 17.6 vs 18, which mostly has little effect, but is a huge boost for Text Classification.
As always F3 results run on the GPU and we see a big difference.
F16 and I8 results mostly run on ANE, and we usually see very little difference either between F16 and I8, or not much beyond the 10% or so from usual tweaks and higher GHz from A16 to A17. There's a lot of variation, sometimes explicable, usually not, but nothing that suggests a pattern of ANE I8 runs at 2x ANE F16.
Certainly nothing that supports these claims.


I continue to hold that the most likely meaning of Apple's claim of doubled TOPs in A17 is basically a redefinition of TOPs in reaction to other companies, not redefining I8 vs F16 but redefining what counts as an "op" to include who knows what other elements that the ANE can perform in parallel with FMAC's - maybe activations?, maybe pooling and other planar ops?, maybe applying some standardized scaling to the timing of some standardized benchmark?

 

[JouniS]

While Apple's made some missteps over the years, and you can argue that the car is a very big one (though I don't think that that's at all obvious), the disastrous picture you paint seems to be very far from the truth.

It's not a disastrous picture. All large organizations are dysfunctional and inefficient, because humanity has not been able to figure out how to coordinate work with a large number of people. Instead of shared goals and efficient coordination, you get politics.

Google is even worse. With them, product decisions are often driven by the effect on the promotion packages of the people responsible for the product. Or consider Nokia back in the day. When they dominated the mobile phone market and didn't have enough external competition, they started encouraging internal competition between product lines. And we all know how well that turned out.

 

[altaic]

I think PCC is a big step

Please tell me you’re referring to proof-carrying code… even if you’re not 🥹

 

[altaic]

the ANE can perform in parallel with FMAC's - maybe activations?

What do you mean by activations? Are you referring to its usage in ML or something else?

 

[altaic]

But in the last few months there has been very little activity on the patent front, and it seems like most published patents were already implemented. There is a bunch of patents that describe ECC memory techniques on Apple Silicon, and also a large amount of advanced NPU-related patents.

I’d love to have a look at those NPU-related patents if you have the links handy.

 

[Confused-User]

This is one of these claims that is endlessly repeated on the internet. But repetition doesn't make it true.
I keep waiting for the evidence of this supposed 2x INT8 throughput and I never see it. But of course the sorts of people who repeat this claim don't bother with pesky details like "evidence".

...well crap.

I thought you were one of the people who answered me about this in the affirmative when I asked last month. OK, I withdraw my statement about doubling INT8 rates. (Though I think I'm correct that that's what the other poster was talking about, in the original comment, regardless of whether or not it's true.)

 

[Confused-User]

It's not a disastrous picture. All large organizations are dysfunctional and inefficient, because humanity has not been able to figure out how to coordinate work with a large number of people. Instead of shared goals and efficient coordination, you get politics.

There's a lot of truth to that, but it's not the only possible mode. Sometimes, not forever, large organizations will defy the odds and do better - sometimes much better. I think there's a lot of evidence that the AS team is in that rare position. Other parts of Apple may be too. Some are definitely not...

 

[leman]

Remember that the patents only get *published* some time after submission :-(

When you look at the past you can see the pattern laid out. But when you try to imagine the future, problem is most of the patents that might be relevant to M4 family are still in processing. (Both being analyzed and I don't know if Apple can ask for "holds" to be put on them, to only be published after some "reasonable' future date, presumably after some relevant product launches.)

Another case you missed is the vector DSP patent. This MIGHT be modem related, but the names on it are mostly the names behind the original ANE patents. So? New functionality for ANE? Or they felt ANE was a solved problem and they'd tackle something new and different?

All very true. I was thinking about the fact that Apple patents often pop up just before the respective product release (seems to be the case for CPU/GPU patents at least). It is entirely possible that I suffer from confirmation bias though.

Right, and I should have been clearer that what I was talking about was the new development that INT8 is doubly fast in the latest silicon. But as you can see, even in this thread, there's a lot of confusion about that.

I hope that one day, we will know more about these things. As Maynard says, there is no clear evidence that there is any difference in INT8 and FP16 execution speed. Furthermore, the expensive part is the binary multiplier, and so far, Apple seems to have been reusing the same hardware to calculate both INT8 and FP16 operations (FP16 has a 10-bit mantissa, so a 10-bit multiplier can feed both). It would be odd if they improved one without improving the other. Now, FP8 could be faster since it only needs a 5- or 4-bit multiplier...

So I agree with Maynard that there is likely something else. For example, some of the newer patents refer to separate specialized neural (convolution) + planar (vector) engines. Maybe this new design has been shipped in A17/M4, and they are adding the throughputs together?

By the way, something I find interesting is that AMD's Strix Point is advertised with a 50 TOPS NPU, but its Geekbench ML results are 20-30% slower than the M3 with its 18 TOPS NPU. Frankly, neural TOPS is an extremely useless measure. It is only informative if the architectures are the same.

I’d love to have a look at those NPU-related patents if you have the links handy.

You can scroll through these; there is a lot of interesting stuff here:

WIPO - Search International and National Patent Collections

patentscope.wipo.int

 

[altaic]

You can scroll through these; there is a lot of interesting stuff here:

WIPO - Search International and National Patent Collections

Appreciated, but unfortunately that link just goes to wipo's search page.

 

[leman]

Appreciated, but unfortunately that link just goes to wipo's search page.

Ah, sorry about that, I was certain they include the query in the link. At any rate, you can use the following query —

FP:(neural) AND ALLNAMES:(Apple)