Apple Silicon deep learning performance

[GrumpyCoder]

I don't do much speech, mostly vision, but... memory. The A5000 runs circles around the M4 Max for compute, but it's limited to 24GB.

Also, it's ironic that Intel MBPs ran so hot, loud and throttled and now we're in the same situation with the 14" MBP (at least throttling as it seems). There were already complains about noise with the M3 Max in the 14". Looks like M4 Pro is the new best choice for 14", but that also means back to a mobile + a desktop Mac instead of using the maxed out MBP for everything.

 

[Gnattu]

Also, it's ironic that Intel MBPs ran so hot, loud and throttled and now we're in the same situation with the 14" MBP (at least throttling as it seems). There were already complains about noise with the M3 Max in the 14". Looks like M4 Pro is the new best choice for 14", but that also means back to a mobile + a desktop Mac instead of using the maxed out MBP for everything.

Back in the Intel days you don't even have a choice to put the same chip into the 13 inch as the 16 inch, and the 13 inch has less than half of the performance of the 16 inch one, and the fact that we can have such Max chip in 14 inch is already a big win.

The maxed out MBP 16 inch does not throttle with M3 Max (with maxed fan) and I think it will not throttle with M4 Max either as the M4 Max actually improved the thermal a bit. So if you want to sacrifice the portability a bit you can get the 16 inch one.

The core problem here is that the Max GPU is too big for 14 inch to run at full clock sustained and unfortunately is the case for machine learning workloads. I personally still prefer 14inch though, because the 16inch is a bit too heavy to carry every day, at least in my opinion it is a bit too heavy.

I'm still excited to have the choice to run 70B models on a 14inch laptop with usable token generation speed, and MBP 14 is still the only choice at the moment due to the aggressive VRAM limit Nvidia is enforcing on its laptop chips.

 

[leman]

Also, it's ironic that Intel MBPs ran so hot, loud and throttled and now we're in the same situation with the 14" MBP (at least throttling as it seems). There were already complains about noise with the M3 Max in the 14". Looks like M4 Pro is the new best choice for 14", but that also means back to a mobile + a desktop Mac instead of using the maxed out MBP for everything.

Well, Intel chips were already heavily throttling when opening a spreadsheet. The fan would go turbo in single-threaded tasks. Throttling with Intel chips meant that the high-end CPU models were often slower than midrange ones. It’s really not the same thing.

 

[theorist9]

Also, it's ironic that Intel MBPs ran so hot, loud and throttled and now we're in the same situation with the 14" MBP (at least throttling as it seems). There were already complains about noise with the M3 Max in the 14". Looks like M4 Pro is the new best choice for 14", but that also means back to a mobile + a desktop Mac instead of using the maxed out MBP for everything.

Sorry to pile on with everyone else (I like your posts!), but I don't believe this is an analogous situation. I checked the Wikipedia table for the last generation of Intel MBP's (Touch Bar, 2016-2020), and the 13" always got a much lower-TDP class of processor (U or NG7-series) than the 15" (H, HQ, or HK-series).

So it's only because Apple has been able to offer the same class of CPU/GPU in the 14" and 16" AS MBP's that we're starting to see significant thermal issues with the maxxed-out (no pun intended) 14".

A more direct analog to the Intel era would be a 14" AS Pro (or even 14" base AS) vs. a 13" U/NG7-series, and a 16" AS Max vs. a 15" H/HQ/HX-series.

Source:

MacBook Pro (Intel-based) - Wikipedia

en.wikipedia.org