MP All Models Apple sleeping on the mac pro is comically bad timing

[leifp]

I was thinking about this the other day. Not the same, yet the same. A Mac Pro would be a magnificent beast, now.

As some note, unified memory is the key, and for that the M series Mac Pro (model of one) was not exactly a step up from the Mac Studio. However, I imagine most people dreaming of a Mac Pro are dreaming of the 2019 machine, which had up to 1.5 TB of RAM available and you could buy aftermarket. And add GPUs. Flexibility was the point of the Mac Pro and the M2 Ultra version never had that beyond PCIe lanes, which are not nothing (but there are not enough of them on Apple Silicon but that’s a design choice not an inherent limitation), but without GPU and RAM expansion they were, and remain, quite limited.

Had Apple spent time and effort they likely could have crafted a system that offered the following, this year:
- an M5 Ultra chip (perhaps it’s the next generation Ultra that will allow the ability to choose more specifically CPU core and GPU core counts, perhaps this one allows it already… tbd) which supported more PCIe bandwidth,
- additional GPUs (Apple’s own? AMDs which are still supported on macOS on the old Intel Macs? nVidia back in the fold [so much money to make even Tim Cook giddy])
- RAM much like the old SLI days (its drawbacks matter in gaming but not much elsewhere) and on-chip cache RAM (L1, L2, L3 et al) in that you’d still have the super-fast unified “on chip” RAM of Apple Silicon but have a secondary cache of slower DDR5 to fill the gaps, like the Fusion Drive did in the days when Apple combined SSDs and HDDs to balance speed and space, a bit like a PHEV can be a “best of both worlds” at present.

All in a machine that did not change on the outside. No garish Frankensteinian towers, no expanding collection of wall plugs to power the beast, no external DAS [unless you wanted], and bandwidth exceeding the dreams of Thunderbolt 5 to share the data. And for the truly ruthless, you could still daisy chain Mac Pros with Thunderbolt to access dizzying levels of RAM that could be expanded when the market for it revives or budgets allow.

I would pay for that. I would never have paid for the M2 Ultra Mac Pro.

 

[SDAVE]

guys no one was buying the M2 Mac Pro, hence the nuke and EOL of the Mac Pro.

It's going to be a very niche market/segment that will buy 128GB-1TB RAM Apple Silicone Mac Pros. They don't even have enough PCIe lanes, so it's a major flop.

 

[Charlie Bonesx]

There is as new wave of people setting up computers locally (both personal and business) for AI inference.

It is striking to me that right before this happens, Apple discontinues the Mac Pro which was perfectly suited for this. We go decades with the Mac Pro having sluggish sales and then the year that it will actually start selling like hotcakes, it's gone!

I guess Nvidia captures the market share with RTX Spark, that they announced a couple of days ago.

For the uninitiated, running a local LLM requires enough VRAM to fully fit it in memory. Models get huge very quickly. Because of the unified memory, this means you can fit much larger models on a mac instead of a traditional machine that has a separate video card (the video card has its own RAM and video cards with a lot of RAM are very, very expensive)

Mac Studio is fully capable

With Apple silicon all ram is shared with the gpu

How much $ is an nvidia card with 256 GB of ram

 

[TechnoMonk]

Mac Studio are Mac Pro. Except the pcie slots, and few who would rack Mac Pro it just wasn’t worth. MP didn’t have big enough market to deviate from unified memory and discrete GPU.

 

[theluggage]

As some note, unified memory is the key,

but without GPU and RAM expansion they were, and remain, quite limited.

GPU expansion is incompatible with unified memory. Also, most of the people begging for dGPU support also seem to be begging for NVIDIA/CUDA which ain't gonna happen if only "because politics".

RAM expansion could still be done with unified memory but that loses the bandwidth advantages of having the RAM mounted directly on the package with short-as-possible traces. Also, when Apple Silicon started, LPDDR RAM had to be soldered direct to the motherboard. There are now press-fit LPCAMM modules - but they don't seem to have taken off (last I looked I couldn't actually find them for sale). Using DDR RAM as cache might be an idea - but doesn't have the advantage of working like VRAM and is subject to the same price rises and shortages as any other RAM.

which supported more PCIe bandwidth,

The Mx Ultra has 32 PCIe lanes (22 free for slots, even that means sharing some with internal peripherals via a switch) - which is only acceptable bcause it doesn't rely on external GPUs. Competing Xeon/Threadripper CPUs have 128 lanes.

Matching that would need Apple to design, produce and tool-up for a new Apple Silicon die just for the relatively tiny Mac Pro market, which would be extortionately expensive - even the current Mx Ultra relies on the economy of using two fused Mx Max dies (as used in MacBook Pros).

You'd end up with a so-so ARM-based tower with a massive "Apple tax" but with no huge advantages over generic PC hardware, limited software compatibility and no "trickle down" of Apple Silicon/Metal-optimised software for Mac small-form-factor, laptop and tablet systems.

- additional GPUs (Apple’s own? AMDs which are still supported on macOS on the old Intel Macs? nVidia back in the fold [so much money to make even Tim Cook giddy])

A Mac Pro with AMD and NVIDIA GPUs (and a hugely expensive custom CPU with enough PCIe to drove them) would turn in similar performance to a Threadripper with exactly the same AMD/NVIDIA GPUs - and that's assuming Apple's drivers were on a par with the Windows/Linux ones. Apple's own GPUs still wouldn't have the "unified RAM" advantage.

I wouldn't rule out Apple producing some sort of "compute module" with an Mx Ultra SoC and PCIe for fast clustering, but that would be vastly different from any Mac Pro that ever existed.

 

[theluggage]

Simple question, how do you make the Mac Pro carbon neutral?

Simple answer: plant some trees or install some renewable energy source that will "save" an equivalent amount of carbon emissions. That's what "carbon neutral" means - whether "carbon neutral" is going to save the planet is a topic for another forum.

Meanwhile, trees are nice, and so is money to improve renewable energy tech.

 

[con2apple]

Apple discontinues the Mac Pro which was perfectly suited for this

I think you have forgotten that the Mac Pro was just a Mac Studio with more space for Storage.

Apple had never any interest to make the Mac Pro a Workstation with upgrade options for GPU.

The Mac Pro was a VW Golf in the body of a Ford Transit.

 

[con2apple]

GPU expansion is incompatible with unified memory. Also

Correction:
Apple makes it impossible.
"Technically incompatible" is not correct

 

[Siliconguy]

Were there ever Mac systems with slotted VRAM?

Way back when. My PowerMac 7500 had four VRAM slots. The beige G3 had a slot for a VRAM expansion card. Note we are talking about 2 to 6 Megabytes of VRAM, not enough for your LLM.

 

[Astuces iOS]

Simple answer: plant some trees or install some renewable energy source that will "save" an equivalent amount of carbon emissions. That's what "carbon neutral" means - whether "carbon neutral" is going to save the planet is a topic for another forum.

Meanwhile, trees are nice, and so is money to improve renewable energy tech.

Carbon neutral for apple means to reduce the pollution of products by 75% and plant trees for the 25%... So if you can't make it 75% carbon neutral, well you need to remove the product...

 

[theluggage]

Correction:
Apple makes it impossible.

No. Having the GPU/NPU separated from the CPU, memory controller and storage controller by a PCIe bus, so they need localised VRAM, is what makes "unified memory" impossible.

 

[seek3r]

@theluggage that was a great video. Thunderbolt has a lot of untapped potential, and so does this studio clustering. But it's no substitute for a real Mac Pro.

I mean, it’s exactly the same as the real MP minus PCIe slots…

This cluster isn't running anything that we couldn't in 2019

We had a CPU/GPU ARM SoC with unified RAM from Apple in a commercially available Mac in 2019? The current machines blow the 2019 MP out of the water for nearly every workload…

and it still costs $40k.

for anyone who actually needs that the $40,000 isnt a problem

Nobody wants this on their desk when Mac Pro is possible and exists.

Most folks who will end up using clusters of studios, professional use, will probably end up rack mounting them in something like sonnet’s xMac racks

The cables aren't secure.

See above

There's an extra layer of management with the cluster vs. a monolithic box.

Honestly not really that much

There's paltry FP64 performance and no way to add more.

That’s fair

I'd also like to see how gracefully it handles a full compute load while gaming at 4k. My old box does it fine.

If you’re buying these machines for gaming you arent the target market. Like, they *can* game, but that’s definitely not what Apple is targeting the studio at, or the now dead MP for that matter

The clustering is a great feature to have, just like target disk mode.

RDMA is a little different from TD mode…

But for today, I would hardly say this solution is perfectly suited. Glad it's an option folks can rig up though - it'll get better I hope. Some part of me clings to hope that they actually have Xgrid 2.0 ready to drop on us. 🤣 - I only got to set that up a few times back in the day. For CRISPR I think it was...

It’d be nice but I think Apple is staying completely out of the HPC side right now. They never were really a major player, and they jettisoned that end of things entirely with the xserves

 

[Unregistered 4U]

There is as new wave of people setting up computers locally (both personal and business) for AI inference.

It is striking to me that right before this happens, Apple discontinues the Mac Pro which was perfectly suited for this. We go decades with the Mac Pro having sluggish sales and then the year that it will actually start selling like hotcakes, it's gone!

I guess Nvidia captures the market share with RTX Spark, that they announced a couple of days ago.

For the uninitiated, running a local LLM requires enough VRAM to fully fit it in memory. Models get huge very quickly. Because of the unified memory, this means you can fit much larger models on a mac instead of a traditional machine that has a separate video card (the video card has its own RAM and video cards with a lot of RAM are very, very expensive)

Are there even any consumer options right now other than Macs that offer 128 G of VRAM? Even if you assume some RAM used by the OS and the application, that’s still well over 90 G of space available. And that’s available in a laptop that rivals the performance of the Mac Pro.

 

[leman]

Well the tooling is mostly Python and C++ or rather CUDA based and pretty agnostic of the OS beneath it. And Microsoft has many interesting projects like MXC for example (in addition to WSL 2).

So one advantage that Windows has (IMHO) is that you can run all these tools within a corporate environment, i.e. with all the controls, authentication etc. that comes with those environments.

Also, a lot of knowledge in big corporations is in Microsoft systems.

So for inference, Windows absolutely is an interesting platform I think (at least for businesses).

I cannot exclude the possibility of Windows increasing adoption for these type of workloads, however, at the current time most "serious" ML work is done on Linux, AFAIK. The framework you linked looks interesting, yet notice how it's targeting *nix systems, because that's what ML engineers work with.

I also wonder what you mean by "inference". It is hard for me to imagine a scenario where business/financial ML models are being run on local workstations. One would think that large corporations that invest in their own inference infrastructure might want a proper local datacenter that is capable of ingesting and processing large amounts of data using complex models. In fact, even if we are talking about local LLM deployment (e.g. for coding), a corporate environment is almost always better off with their own data center to increase model capability and pool resources, rather than running these LLMs locally.

 

[leman]

I would pay for that. I would never have paid for the M2 Ultra Mac Pro.

You would? How much? Note that these kinds of machines exist — they are Nvidia's datacenter ML solutions, and their cost far exceeds that of regular workstations. Fast switches, large chips, and a lot of RAM are not cheap.

There are significant engineering challenges to making what you envision possible. For example, Apple chips don't have enough high-speed I/O to drive high-performance external GPUs, because unified memory controller take a lot of die edge space (and that's where the I/O is routed from). Traditional server CPUs don't have this problem because they don't use high-bandwidth RAM. Now, this is potentially solvable, of course (and Apple has filed relevant patents!) but the solution would be prohibitively expensive.

Looking at this realistically, your proposal boils down to one of the following options:

- create a server-grade Apple Silicon system with more performance and RAM capabilities (which would drive R&D and manufacturing costs to a point that most customers likely won't find attractive)
- create a dedicated expandable system that can be equipped with third-party solutions (which would break Apple's fundamental programming model and fragment the software ecosystem for the sake of a niche market)
- create a separate line of computers targeting workstation market and potentially ship them with Windows/Linux

Overall, it doesn't seem likely that there is much business sense for them going that route. In fact, Apple's own custom datacenter use modular systems that leverage multiple "small" managed SoC boards rather than large systems like you describe. It just appears to be a more economical solution. And for the end customer interested in local inference, the Studio with Ultra chips offers plenty of RAM and compute to power many relevant applications. The M5 Max is already equivalent to Nvidia Spark for half-precision work (Nvidia of course is massively ahead for fixed-precision and quantized matrix compute). Future systems will bring new capabilities.

 

[con2apple]

No. Having the GPU/NPU separated from the CPU, memory controller and storage controller by a PCIe bus, so they need localised VRAM, is what makes "unified memory" impossible.

No one has claimed that it would still be unified memory. So that’s a straw man argument on your part.
The point is simply that, technically speaking, Apple could certainly integrate the interface to connect a GPU.

Apple could also add an interface for dedicated RAM. Yes, that would be a bit more complicated than the current setup and it would also be a bit more expensive. But it would be possible.

And it could have turned the Mac Pro into a workstation instead of a dust catcher.

 

[keksikuningas]

I think you have forgotten that the Mac Pro was just a Mac Studio with more space for Storage.

Apple had never any interest to make the Mac Pro a Workstation with upgrade options for GPU.

The Mac Pro was a VW Golf in the body of a Ford Transit.

that almost exists by the way. VW Transporter VR6 🙂 just a car tangent from me

and yeah i agree, Apple would rather die than let you put a GPU in the AS MP. iirc the M2U physically lacks things to properly support a GPU even if there were drivers

 

[leman]

No one has claimed that it would still be unified memory. So that’s a straw man argument on your part.
The point is simply that, technically speaking, Apple could certainly integrate the interface to connect a GPU.

Apple could also add an interface for dedicated RAM. Yes, that would be a bit more complicated than the current setup and it would also be a bit more expensive. But it would be possible.

And it could have turned the Mac Pro into a workstation instead of a dust catcher.

Quite a lot of things are technically possible, what matters here is whether they are good business decisions.

To draw on the ever popular car analogy — technically, Ferrari could make all of their cars family sedans with a large trunk, but would that be a good choice for their primary market and branding? And before you point out that this analogy is flawed, consider that enabling external GPUs in one niche Mac models would mean substantial and dramatical changes to the software model of every single Mac. Apple delivers a streamlined software model with strong guarantees. Third-party GPUs would completely destroy this.

 

[BSDnostalgia]

the M3 Ultra Mac Studio is also perfectly suited to local LLM work

Excellent video. I'm usually disappointed by the low information density of such videos, but, this one is very informative. The only point I would argue against is that I think it would make perfect sense for Apple to create an "Xserve" version with QSFP ports. Apple would make a lot of money and friends both. They would just have to hide it from people who might be frightened by it.

 

[BSDnostalgia]

People love to get all up in arms about which Macs they think Apple should be building, but we're talking about a 7.6% segment of their revenue. And the vast majority of that 7.6% is from MacBooks. That leaves, what, maybe 2% of their overall revenue coming from these desktops they’re doomed because they didn’t bring to market? Please.

The bean-counter approach to anything that isn't for sale at Walmart. MBAs like to keep things simple to manage. That doesn't mean that companies like Apple are not making a profit on things like MBPs. Just a little more complex to manage.

 

[jakey rolling]

Quite a lot of things are technically possible, what matters here is whether they are good business decisions.

To draw on the ever popular car analogy — technically, Ferrari could make all of their cars family sedans with a large trunk, but would that be a good choice for their primary market and branding? And before you point out that this analogy is flawed, consider that enabling external GPUs in one niche Mac models would mean substantial and dramatical changes to the software model of every single Mac. Apple delivers a streamlined software model with strong guarantees. Third-party GPUs would completely destroy this.

Pretty sure Enzo's buddy Ferrucio would tell him that one can actually produce a good sports car and also make excellent utility vehicles (you know, like farm tractors) at the same time.

 

[BSDnostalgia]

I could very well imagine that happening. RTX Spark has two huge advantages: CUDA and Windows. The latter might be controversial, but in an enterprise setting, this is absolutely a plus.

Depends. The internet runs on *nix you know. CEOs think that ordinary people use Windows, but, plenty of ordinary people use Macs. A few services still use BSD Unix directly (e.g. FreeBSD). All *nix. And, plenty of expertise in administering *nix out there, including in most of Nvidia's institutional customers.

CUDA, OTOH, you have a point. It is the defacto API.

Those Mac Studio setups are nice for proof of concepts or personal projects or maybe small businesses, but larger companies will appreciate all the controls and management you get with Windows.

You have to be joking. Only CEOs think it is cheaper to operate and manage Windows. The advantage of Windows is that it is familiar to managers in large companies and their minions. That, and, Microsoft's iron grip on Excel. Those are big advantages, but, could be overcome if these companies were actually as cost-conscious as they say they are.

 

[SDAVE]

Mac Studio are Mac Pro. Except the pcie slots, and few who would rack Mac Pro it just wasn’t worth. MP didn’t have big enough market to deviate from unified memory and discrete GPU.

You can rack mount Mac Studio + Add PCIe expansion via rackmount via Thunderbolt. Frankenstein model, but I've seen companies do it.

 

[RedWeasel]

You have to be joking. Only CEOs think it is cheaper to operate and manage Windows. The advantage of Windows is that it is familiar to managers in large companies and their minions. That, and, Microsoft's iron grip on Excel. Those are big advantages, but, could be overcome if these companies were actually as cost-conscious as they say they are.

Not only that. It's also things like access control, auditing, data protection and governance. E.g. centralized control over who has access to what data, which agents, which models etc.

These are areas where Microsoft has a very strong foothold in the enterprise world (whether or not justified is another question) with services like Entra or Purview. Apple basically has nothing.

Hyperscalers like AWS or GCP have that as well in their cloud offerings, but here MS has an advantage to have both a cloud offering and a client system.

That's why I said Mac-based setups are nice for small businesses or personal projects, where the above factors are usually not important.

 

[RedWeasel]

A few services still use BSD Unix directly (e.g. FreeBSD). All *nix.

PS: I started using FreeBSD I think with a 2.2.x or 3.x version, or both. It's been a while. I think I got one set of the Walnut Creek CD-ROMs and also the 2nd Edition of Greg Lehey's The Complete FreeBSD (the one with the green title), I'm not sure which version came with that. I learned a lot about operating systems with that, even though I professionally was more into Windows NT then (mainly because there were more jobs for it where I lived).

The Complete Freebsd - Greg Lehey: 9781571762276 - AbeBooks

FreeBSD is a full, professional quality, UNIX - compatible operating system. FreeBSD is based on Berkeley 4.4 BSD, developed by the University of California, Berkeley, and its contributors. It comes with industry standard X Window system (X11R6) for the PC, XFree86, along with both the KDE and ...

www.abebooks.com

In fact, FreeBSD was the reason that I became interested in Mac OS X when the Public Beta was out in 2000.

So BSD does indeed make me nostalgic a bit 😉

I even still have this somewhere in a bookshelf: https://www.mckusick.com/csrg/