Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

leman

macrumors Core
Oct 14, 2008
19,516
19,664
Yep, I actually think SLC cache is associated with memory controller as well... That's why by reducing memory bus, the amount of SLC should be reduced...

You are correct that SLC cache is essentially a memory controller cache (Apple even refers to it as such in their CPU architecture manual). They can always increase the individual caches to compensate for the narrower bus. Of course, it requires tweaking the individual blocks.


For example, the latest non-Pro iPhone still has USB 2.0 data speeds

I was under impression this is the case because it uses A16 and that chip does not have an USB3 controller, or am I wrong?

Someone who knows about what it takes to run local on-device AI can help inform us about RAM requirements. (Not me)

Well, Apple has published some research on reducing RAM consumption by the LLM. If I remember correctly they propose a scheme that combines the SSD and the RAM with aggressive weight compression and smart caching to achieve good performance without needing to load the full model into RAM.

Of course, training is an entirely different topic. I agree with others who said that Apple is unlikely to pursue large model training on Apple Silicon. It's not something you can do on a single workstation anyway, you need a specialized datacenter for that. But maybe they want to make Mac Pro capable of training smaller models (several hundred million to few billion parameters), that would be very useful for many practical applications. Or maybe even fine-tuning large models.
 

hovscorpion12

macrumors 68040
Sep 12, 2011
3,043
3,120
USA
I was under impression this is the case because it uses A16 and that chip does not have an USB3 controller, or am I wrong?

Correct. The 15 and 15 Plus have USB-C up to USB 2.0 speeds at 480 Mbps, whereas the 15 Pro/Max have USB 3.0 up to 10 Gbps
 

hovscorpion12

macrumors 68040
Sep 12, 2011
3,043
3,120
USA
Of course, training is an entirely different topic. I agree with others who said that Apple is unlikely to pursue large model training on Apple Silicon. It's not something you can do on a single workstation anyway, you need a specialized datacenter for that. But maybe they want to make Mac Pro capable of training smaller models (several hundred million to few billion parameters), that would be very useful for many practical applications. Or maybe even fine-tuning large models.

I have seen evidence of the M2 Ultra/M3 Max perform large LLMs on device. M2 Ultra does have a 32-Core NPU at 31.6 TOPs.

Even Apple themselves have claimed it.

"Finally, the 32-core Neural Engine is 40% faster. And M2 Ultra can support an enormous 192GB of unified memory, which is 50% more than M1 Ultra, enabling it to do things other chips just can't do. For example, in a single system, it can train massive ML workloads, like large transformer models that the most powerful discrete GPU can't even process because it runs out of memory."

WWDC 2023 — June 5
 

leman

macrumors Core
Oct 14, 2008
19,516
19,664
I have seen evidence of the M2 Ultra/M3 Max perform large LLMs on device. M2 Ultra does have a 32-Core NPU at 31.6 TOPs.

Even Apple themselves have claimed it.

"Finally, the 32-core Neural Engine is 40% faster. And M2 Ultra can support an enormous 192GB of unified memory, which is 50% more than M1 Ultra, enabling it to do things other chips just can't do. For example, in a single system, it can train massive ML workloads, like large transformer models that the most powerful discrete GPU can't even process because it runs out of memory."

WWDC 2023 — June 5

That's not large LLMs. Large LLMs is something like GPT4 that requires 10'000 V100-class GPUs running for half a year. Yes, M2 Ultra has more RAM than any other workstation GPU, which does allow it to do some things that other GPUs will have difficulty with. But let's not overestimate the capabilities here.

P.S. Not to mention that workstations can have multiple GPUs. One of our labs bought a workstation recently, which I believe has 8x Nvidia A100 with 80GB each — that's 720GB of GPU working RAM overall.
 
Last edited:
  • Love
Reactions: hovscorpion12

MrGunny94

macrumors 65816
Dec 3, 2016
1,148
675
Malaga, Spain
If the M4 Pro starts at 24GB I won't even need to upgrade as my memory needs are around 19-20GB! :) Let's see if these can finally bring an OLED display, then I'll definitely upgrade from my M2 Pro.

From the specs itself LPDDR6 does look like an insane upgrade to M1+ users.
 

TigeRick

macrumors regular
Original poster
Oct 20, 2012
144
153
Malaysia
You are correct that SLC cache is essentially a memory controller cache (Apple even refers to it as such in their CPU architecture manual). They can always increase the individual caches to compensate for the narrower bus. Of course, it requires tweaking the individual blocks.

Thanks for confirming my theory. Didn't know Apple actually refers it as memory controller cache. And I don't think Apple will increase the amount of SLC per channel cause 92 billion are a lot for monolithics SoC. In fact, TSMC only expecting SoC to hit 100 billion in upcoming N2 process.

If my calculation is correct, upcoming M4 Max should have 48MB of SLC (reduced from 64MB). Meanwhile, M4 and M4 Pro are getting SLC bump...


 

leman

macrumors Core
Oct 14, 2008
19,516
19,664
Thanks for confirming my theory. Didn't know Apple actually refers it as memory controller cache. And I don't think Apple will increase the amount of SLC per channel cause 92 billion are a lot for monolithics SoC. In fact, TSMC only expecting SoC to hit 100 billion in upcoming N2 process.

If my calculation is correct, upcoming M4 Max should have 48MB of SLC (reduced from 64MB). Meanwhile, M4 and M4 Pro are getting SLC bump...

They will soon be migrating to multi-die designs, with cache and memory controllers likely residing on their own N5 die.
 

TigeRick

macrumors regular
Original poster
Oct 20, 2012
144
153
Malaysia
They will soon be migrating to multi-die designs, with cache and memory controllers likely residing on their own N5 die.
I am not so sure Apple will move to chiplet soon. M5 series supposedly fabbed at N2 process which has plenty headroom for PPA. We shall see..
 

TigeRick

macrumors regular
Original poster
Oct 20, 2012
144
153
Malaysia
Standard 24Gb (3GB) Memory Die for iPhone 16 Pro series



According to above two articles, JEDEC will finalize LPDDR6 in Q3 this year. At the same time, Qualcomm upcoming 8G4 which is supposedly launching in Q3 this year also will feature LPDDR6. That's mean Samsung and SK Hynix have been making LPDDR6 memory die at least one year earlier. JEDEC specification is more like formal announcement. This is in line with Samsung roadmap which I posted in the front page. That's mean Samsung has been making 24Gb 6.4GBps memory die since beginning of 2023 should be true.

Let's say upcoming iPhone 16 Pro series will occupy 40% of total volume; that's mean iPhone 16 Pro will need 20 million memory chips per quarter. Yes, that's a lot compared to 8G4 which I estimated will need 3 million memory chip per quarter. Yes, iPhone volume is crazy huge but if Samsung has been making LPDDR6 since 2023, I think they will provide enough volume for Apple. Don't forget Apple has the purchasing power and volume, it is actually other OEM like Qualcomm's partner Xiaomi need to worry about getting enough quantity cause majority of LPDDR6 memory chip will be consumed by Apple.

In case people don't know about A17 Pro's GPU performance; Apple has been lacking compared to Android phone. The reason is due to usage of LPDDR5. With limited memory bandwidth, A17 Pro's GPU can only hit 2.15TF, far lower than 8G3 and D9300 which are benefited from LPDDR5X. Why don't Apple use LPDDR5X, cause Apple is waiting for LPDDR6. As listed, EUV-based LPDDR6 is cheaper per bit while providing double memory bandwidth. That's why Apple is waiting until iPhone 16 Pro to use it.

Yes, I am pretty sure upcoming iPhone 16 Pro will feature 12GB LPDDR6 and it is going to get biggest boost in GPU performance. I am expecting 50% bump in FP32, let's see...

FYI, there is leaks about upcoming A18 Pro's GPU performance, A18 Pro shown around 38% better FPS in GFXBench's Aztec Ruins High Tier Offscreen benchmark (1440p). Go figure...:cool:
 
Last edited:
  • Like
Reactions: Mac_fan75

TigeRick

macrumors regular
Original poster
Oct 20, 2012
144
153
Malaysia
Custom 32Gb (4GB) Memory Die for Mac lineup

As for Mac, Apple will ask Samsung to custom make 32Gb memory die featuring 96-bit memory bus. I know some of you are not convinced about memory bus. Just ignore it for the moment, the most important is Apple will use 4GB memory die in order to hit 512GB memory size in M4 Ultra as leaked by Mark Gurman. I believe him becuase Samsung roadmap has shown 32Gb will be manufactured in the beginning of this year. And Mac only needs about 5-7 million memory chips per quarter. I believe initially most of the production of 32Gb will be consumed by Apple as well.

As listed in the table of frontpage, LPDDR6 actually provide at least 50% extra bandwidth while requires lesser amount of memory die. This is exactly what Tim wants...So much so until Apple will completely reset whole lineup as leaked by Gurman as well.
 

TigeRick

macrumors regular
Original poster
Oct 20, 2012
144
153
Malaysia
Good analysis. One minor correction that I don't think impacts your thesis - his original quote in his subscription letter (which everyone quoted to varying degrees of accuracy) did not say the entire line would be out within a quarter time period. It will still be close to a year. He said in his newsletter:

Here are the Mac models Apple is working on and when it expects to introduce them (as always, release plans could shift):

  1. A low-end 14-inch MacBook Pro with the M4, coming around the end of 2024.
  2. A 24-inch iMac with the M4, also expected around the end of the year.
  3. New 14-inch and 16-inch high-end MacBook Pros with M4 Pro/Max chips, due between the end of 2024 and early 2025.
  4. A Mac mini in both M4 and M4 Pro configurations, coming between the end of 2024 and early 2025.
  5. New 13-inch and 15-inch MacBook Airs, slated for around spring 2025.
  6. A Mac Studio with a high-end M4 chip, coming around the middle of 2025.
  7. A Mac Pro with an M4 Ultra chip, due in the second half of 2025.
Thanks for detailed lineup, it's time for my analysis with updated information. As I am more convinced that 12GB LPDDR6 will be standard in upcoming M4 SoC. One question comes to mind is will Apple increase the base storage as well since it is going to be full reset? Apple has been using 8+256 configuration since M1 period, isn't it the best time to upgrade to 12+512???? I believe so, cause by the time next year, we should see army of ARM notebook with 64-bit memory bus costing below US$1K. And Apple know it, that's why Apple has to completely upgrade the lineup including Mac Mini.

That's mean M2-based Mac Mini + MBA and M3-based MBA 13+15 will be fully replaced by M4-based Mac Mini and MBA 13+15 with 12+512 configuration. And I will let you guys guess the pricing of each model....:cool:
 
Last edited:
  • Like
Reactions: M1956

MrGunny94

macrumors 65816
Dec 3, 2016
1,148
675
Malaga, Spain
Custom 32Gb (4GB) Memory Die for Mac lineup

As for Mac, Apple will ask Samsung to custom make 32Gb memory die featuring 96-bit memory bus. I know some of you are not convinced about memory bus. Just ignore it for the moment, the most important is Apple will use 4GB memory die in order to hit 512GB memory size in M4 Ultra as leaked by Mark Gurman. I believe him becuase Samsung roadmap has shown 32Gb will be manufactured in the beginning of this year. And Mac only needs about 5-7 million memory chips per quarter. I believe initially most of the production of 32Gb will be consumed by Apple as well.

As listed in the table of frontpage, LPDDR6 actually provide at least 50% extra bandwidth while requires lesser amount of memory die. This is exactly what Tim wants...So much so until Apple will completely reset whole lineup as leaked by Gurman as well.
Okay, then we can suppose the 'Pro' lineup can start at either 24GB - 32GB? Thanks for the great post and information.

I'm looking forward to LPDDR6, even more so across gaming handhelds.
 

TigeRick

macrumors regular
Original poster
Oct 20, 2012
144
153
Malaysia
Okay, then we can suppose the 'Pro' lineup can start at either 24GB - 32GB? Thanks for the great post and information.

I'm looking forward to LPDDR6, even more so across gaming handhelds.
Upcoming M4 Pro should start with 24+1TB configuration. It is good upgrade for your usage. Luckily you didn't buy M3 Pro MBP; the more I think about M3 Pro is really a stepping stone before LPDDR6 arrives.

Which gaming handheld you are interested?? Upcoming ASUS ROG Ally2 should feature 128-bit LPDDR5X...
 

leman

macrumors Core
Oct 14, 2008
19,516
19,664
In case people don't know about A18 Pro's GPU performance; Apple has been lacking compared to Android phone. The reason is due to usage of LPDDR5.

The reason is that Qualcomm designs their GPUs to be fast on simple graphics-related tasks while Apple makes general-purpose GPUs that can run advanced workloads. Qualcomm GPUs are atrocious on compute. More RAM bandwidth would of course be very welcome.



FYI, there is leaks about upcoming A18 Pro's GPU performance, A18 Pro shown around 38% better FPS in GFXBench's Aztec Ruins High Tier Offscreen benchmark (1440p). Go figure...:cool:

Could you provide a link? I couldn’t find anything besides the usual traffic-generating fake news websites.
 

TigeRick

macrumors regular
Original poster
Oct 20, 2012
144
153
Malaysia
The reason is that Qualcomm designs their GPUs to be fast on simple graphics-related tasks while Apple makes general-purpose GPUs that can run advanced workloads. Qualcomm GPUs are atrocious on compute. More RAM bandwidth would of course be very welcome.





Could you provide a link? I couldn’t find anything besides the usual traffic-generating fake news websites.
It is actually from Chinese Weibo, here is the link in English

 

MrGunny94

macrumors 65816
Dec 3, 2016
1,148
675
Malaga, Spain
Upcoming M4 Pro should start with 24+1TB configuration. It is good upgrade for your usage. Luckily you didn't buy M3 Pro MBP; the more I think about M3 Pro is really a stepping stone before LPDDR6 arrives.

Which gaming handheld you are interested?? Upcoming ASUS ROG Ally2 should feature 128-bit LPDDR5X...
I’m a Linux user so I’ll stick to the Steam Deck 2 ;) I love my OLED but need more performance on the go!

Yeah I think for me 24GB should be more than enough especially since I’m using Docker on the cloud these days.

I have the base M2 Pro model but on heavy work days I can see the graph turning orange to reddish with 50-55% memory pressure
 

TigeRick

macrumors regular
Original poster
Oct 20, 2012
144
153
Malaysia
I’m a Linux user so I’ll stick to the Steam Deck 2 ;) I love my OLED but need more performance on the go!

Yeah I think for me 24GB should be more than enough especially since I’m using Docker on the cloud these days.

I have the base M2 Pro model but on heavy work days I can see the graph turning orange to reddish with 50-55% memory pressure
Yep, if you are waiting for SD2, then it should come with 64-bit LPDDR6. The peak performance may not as high as Ally2 but the battery life should be better...The only thing is you have to wait until 2026 according to Valve
 

leman

macrumors Core
Oct 14, 2008
19,516
19,664

Thanks. I would take these with a grain of salt. The 3'570 GB6 score sounds too high, 2'822 is unrealistically low. A-series usually show year-to-year improvement of 250-300 GB6 points. So the next chip should have around 3'200 in GB6.

The higher leaked score would suggest a two-year-like jump. I mean, it is not entirely impossible, if Apple feels pressured by new ARM chips they might do this, and it is also true that larger tweaks to their CPU core is likely incoming. The A17 Pro in many ways feels like an experimental chip, they made the architecture slightly wider, but it seems like the fine-tuning is not yet complete.
 
  • Like
Reactions: Chuckeee

DrWojtek

macrumors regular
Jul 27, 2023
187
401
i think here you lost my credibility and probably from others too
some rumors says that iphones pro will have 256 basic ssd +12 gb lpddr6 ?your hopes are way to high, that leap would be too high
If it happens, its due to China. The competition is fiercer there. More people are likely to buy Android, meaning Apple has to give customers more.

Upping storage and RAM is cheap, the only downside is there’ll be less people who buys the upgraded options with huge margins. I think these people are relatively few.

Let’s hope it’s all true. If it comes with the LYT sensor, what a year to upgrade. iPhone 16 Pro and M4 mac mini.
Especially since the SE in 2025 will tank the value of all non-Pro models. Better upgrade before it happens.
 

MRMSFC

macrumors 6502
Jul 6, 2023
371
381
Well, Apple has published some research on reducing RAM consumption by the LLM. If I remember correctly they propose a scheme that combines the SSD and the RAM with aggressive weight compression and smart caching to achieve good performance without needing to load the full model into RAM.
Man, the lengths Apple will go to be stingy with RAM!

Of course, training is an entirely different topic. I agree with others who said that Apple is unlikely to pursue large model training on Apple Silicon. It's not something you can do on a single workstation anyway, you need a specialized datacenter for that. But maybe they want to make Mac Pro capable of training smaller models (several hundred million to few billion parameters), that would be very useful for many practical applications. Or maybe even fine-tuning large models.
If I were to predict where Apple would go with AI training on their hardware, it’s probably gonna be small models on a single machine, or dedicated towards things like image and music generation. Y’know, stuff that helps individual “creative” types.

I would still like to see something along the lines of a Mac Pro accelerator card with their neural cores on it. That would be interesting at least.
 

leman

macrumors Core
Oct 14, 2008
19,516
19,664
I would still like to see something along the lines of a Mac Pro accelerator card with their neural cores on it. That would be interesting at least.

What kind of applications do you think would benefit from it? How would an accelerator card be better than making the GPU more flexible for training? If you go the accelerator route, you need fast interconnect, and if you have a fast interconnect, why not simply allow multiple SoC boards in the system?
 
  • Like
Reactions: altaic and SBeardsl

deconstruct60

macrumors G5
Mar 10, 2009
12,493
4,053
While this is true for some models, the Android phones with copious RAM aren't the ones moving in volume. In Q4 2023, the best-selling Android phone was the Galaxy A54, which starts at 4GB of RAM. The Galaxy S line, that starts with 8GB, isn't even in the top 10.

Sorry. I meant to put a 'some' Android models ( $150 priced models are going to have tough time maximizing RAM). But generally Android+JDK apps tend to have a higher memory footprint. There is more pressure on Android systems to add memory at all the price levels.

As for the A54 there are multiple models.
"...
Internal128GB 4GB RAM, 128GB 6GB RAM, 128GB 8GB RAM, 256GB 6GB RAM, 256GB 8GB RAM
..."


As 34 same thing.
"...
Internal128GB 4GB RAM, 128GB 6GB RAM, 128GB 8GB RAM, 256GB 6GB RAM, 256GB 8GB RAM
..."


That 'starts at' capacity is trying to hit an overall system price point at least as much as it is RAM availability. Most Android vendors don't get to put an "Apple Tax" (i.e., margin pump ) on their system prices.

And if go back on that page to some Q2 metrics the S2x systems tend to show up. By Q4 it fades because the 'refresh' is coming relatively soon. Everyone in the Android market doesn't release their top end smartphones in exactly the same month every year ( it ends to vary by vendor). It is Apple vs Android (in the aggregate) not Apple vs Samsung that is RAM consumption volume at the whole market level.

A contributing factor is Apple's static September launch times also. If next gen memory ships in 2H 202x then it won't make the cut. Doesn't matter how much the memory vendors can/cannot make. Apple needed to have the parts in April-May to start ramping to have phones done by July-Sept to build inventories to cover the initial demand bubble. No fab vendor "has to" roll out a generally usable technology solely to iPhone's technically arbitrary timeline. ( there is zero technological grounded basis there. It is mainly just the 'corne' that Apple has painted themselves into the corner on. ). Periodically Apple is just going to be out of phase with technology that doesn't move on exactly 12 month cycles.

LPDDR3 May 2012

LPDDR4 August 2014
LPDDR5 February 2019
etc.

Absolutely no 12 month cycle there at all. The cycle for the A-series fab improvements is also on their own schedule not completely beholding to the iPhone inventory build cycle. Other competitors who are not slavishly hooked to September are going to have other options for launch dates.

At the iPhone's scale, doubling RAM capacity could have significant market implications.

Doubling if the is zero increase in memory die density yes. But if the memory die wafer density doubles then it is exactly the same number of dies needed. Apple doesn't have 'double' to do any kind of increase at the base level. ( the Pro and 'regular' iPhone are gapped by 2GB . )

If they just follow the tech it isn't a 'volume' problem. Apple pushes 'economies of scale' ( e.g., same component part into several products ) to push deeper discounts to claw out higher margins. It is to buy a larger subset of what the memory vendors are selling to wrangle a better deal. Or to get some semi-custom package/compoent at the general market pricing (still wrangling a better deal).

Admittedly, this doesn't mean much for the Mac, but it holds a little bit of water for the phones.

There is a little water there. But it is also overblown as a misdirect from getting better deals so they can pass the money around inside of Apple.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.