Apple dedicated GPU

[diamond.g]

It’s the same thing really, if Nvidia can build a huge GPU and give it a 2+ghz clock why can’t Apple?

Poor ROI for Apple to do that for a niche of a niche?

Nvidia sells those "fat" GPU's for stuff other than rasterization; notice how their GTC keynote was ten minutes graphics vs an hour talking about AI, Machine Learning and Autonomous Cars. Nvidia knows what industry "butters their bread".

 

[diamond.g]

This wasn't the accusation made here. The charge is that Nvidia focused on selling their gaming GPUs to miners over gamers. It's not true.

oooh, yeah Nvidia themselves didn't sell direct to miners, but (some of) their partner AIB's definitely got "caught" doing that.

 

[leman]

Poor ROI for Apple to do that for a niche of a niche?

Is desktop that much of a niche? I mean, it makes sense that Apple focuses on laptops first, but it would also be nice if their desktops were competitive. Vertical scaling via a larger clock range sounds like a straightforward way to achieve this. In other words, what I am proposing here is to make a mobile-first product but also try to achieve good overclockability for desktop use.

 

[diamond.g]

Is desktop that much of a niche? I mean, it makes sense that Apple focuses on laptops first, but it would also be nice if their desktops were competitive. Vertical scaling via a larger clock range sounds like a straightforward way to achieve this. In other words, what I am proposing here is to make a mobile-first product but also try to achieve good overclockability for desktop use.

I mean unless Apple plans to offer these beefed up GPU's across all 4 desktop products and not just the Mac Pro, I would think it would be a niche of a niche; I mean how many Mac Pros do they really sell and would it be enough to cover the eng costs?


Not saying they shouldn't do this, I would love for them to rev custom stuff like this. Especially if they offer it across the board at BTO only maybe?

 

[senttoschool]

oooh, yeah Nvidia themselves didn't sell direct to miners, but (some of) their partner AIB's definitely got "caught" doing that.

Well yes. But it's popular to hate on Nvidia on the internet.

 

[senttoschool]

Poor ROI for Apple to do that for a niche of a niche?

Nvidia sells those "fat" GPU's for stuff other than rasterization; notice how their GTC keynote was ten minutes graphics vs an hour talking about AI, Machine Learning and Autonomous Cars. Nvidia knows what industry "butters their bread".

Apple is not in the business of selling discrete GPUs to other parties. Nvidia is.

The only way I see Apple turning a profit on any custom Mac Pro SoC is if they also plan to create Apple Silicon Cloud:

You're Apple. You just built a 40-core SoC for Mac Pro. Now what?

Mark Gurman is saying that Apple is working on a 40-core SoC for the Mac Pro for 2022. You're Tim Cook, sitting in his nice office, looking at how much money you just spent to make this giant SoC for a relatively small market. In fact, you have to do this every year or every two years to keep...

forums.macrumors.com

 

[iPadified]

I think the only reason for building a competitive ASi compute powerhouse using custom Apple GPU is to pull off a marketing stunt against NVIDIA. I might be wrong, but even a single M2/3 Extreme will not be sufficient to be competitive at the very edge.

 

[singhs.apps]

-Delete-

 

[singhs.apps]

building a competitive ASi compute powerhouse using custom Apple GPU is to pull off a marketing stunt against NVIDIA. I might be wrong, but even a single M2/3 Extreme will not be sufficient to be competitive at the very edge.

They already tried it once.

Poohpooing the ‘discrete’ approach by talking about US power limits (hey, let the market decide) smacks of defensivenes cos same discreets are stomping their efforts all over the place…and they got caught lying trying to square up against them.

As some have been saying, decoupling the GPU cores from the CPU cores (same die approach) and pairing them via ultra fusion may work out better.

Also you can still get a ‘discreet’ GPU compute unit that isn’t driving your displays, games or viewports. It just won’t be plugged in via PCI-e but rather another ultra fusion connection that isn’t part of the main SOC but can access the memory same as the main GPU.
You order the configurations as per your needs at the time of purchase.

This can extend to the larger Mac ecosystem too, albeit with fewer cores.

 

[leman]

I mean unless Apple plans to offer these beefed up GPU's across all 4 desktop products and not just the Mac Pro, I would think it would be a niche of a niche; I mean how many Mac Pros do they really sell and would it be enough to cover the eng costs?

What I had in mind definitely won’t be just Mac Pro exclusive feature but common to all desktops. Mac Pro would just be capable of hosting a bigger system. But already the Studio can probably dissipate 200W of power while a Mini can do 60-70…

 

[deconstruct60]

he amount of wish casting is remarkable, and then some people get angry when the genie doesn't appear. Apple literally tells us their plans, yet some folks don't like the answers, so they fill in blanks because it doesn't fit the narrative that they personally desire. Hope springs eternal, I suppose.

The pot calling kettle black.... same boat with the "Apple is doing dGPU" thing with the post #26 in this thread. You keep push the Apple dGPU thing up the hill with the same misrepresentation of other threads, even after shown where you are doing so. How is that not 'wish casting'.

You're thinking of this.

View attachment 2080079

More of his thoughts in the thread I linked. Keep in mind that this guy is Apple's Director of GPU Architecture. He's currently roasting Nvidia's new space heaters on Twitter. Yet, despite what the guy who is literally in charge of Apple Silicon's GPU is saying, some still insist that Apple is going to use third-party GPUs.

And this proves what point? That he is being relatively clueless from a Mac Pro perspective?

Mac Pro 2009-2012 -- power supply 980-1000 W



https://www.macpartsonline.com/661-...0-watts-for-mac-pro-2012-2010-2009-a1289.html


Mac Pro 2019 --- power supply 1400W

" Power Supply​

1.4 kilowatts
..."

Mac Pro (2019) - Technical Specifications - Apple Support

support.apple.com

A historic range of Mac Pro BTU consumption in standard configurations sold by Apple.

Mac Pro power consumption and thermal output (BTU/h) information - Apple Support

Learn about the power consumption and thermal output of Mac Pro computers.

support.apple.com

PowerMac G5 --- power supply 450W


So when Mac Pro folks are expecting a dGPU in a approximately 1KW system like the Mac Pro has been for a decade (minus the bumpy excursion into MP 2013 and retrograde back to 450W from system over a couple of decades ago) , this guy is basically agreeing with them. [ Yes, if only put the GPUs from the 2006-2010 era into a Mac Pro the consumption was way lower. The performance was way, way ,way lower too. In 2022, that lower performance threshold isn't gong to be competitive. ]


The only way his point is valid on the "Mac Pro" is if Apple chops the power of the Mac Pro about in half (450-700W constraints applied to the system).

It is a market that isn't going away. Even in that twitter thread. If actually go back and read the thread that post comes from he is more so talking about that is 'bad' to put a dGPU in a laptop. If trying to say under 200W then yes , the point being made here is on much more solid ground. Over 600W though it is wishy-washy. I suspect he picked 1K as the point where it was pretty clear, but I doubt there is some magic hard property there. As he states in that thread there are industry norms and inertia that will carry dGPU along ( even the big desktop replacement laptop space )


Now Apple's 'required' iGPU in a Mac could cause the max system power supply to fall back under 900W and still offer the same performance with a single upper mid-range GPGPU. But won't cover duals of modern upper mid range.
One rumor for the Mac Pro was that Apple was going to make it "half size". If Apple removes one MPX bay sized chunk from the Mac Pro because that GPU zone is covered by the iGPU then that is quite plausible. But making them both Bays go away completely really isn't if looking to say competitive with systems sticking with the 1400W limit. ( dual 4090s could have problems with 1400W limit with a CPU that also is a high consumer. Duals in the 4080 range of products (and AMD 7800 and future) would present problems in a market that the GPU guru says isn't going away.

For better or worse, Apple is going all-in on their own solutions, we can either adapt to that new reality, or plug our ears to what is, in my opinion, plainly obvious using Apple's own statements from the executives in charge of these projects. I don't necessarily agree with all of Apple's decisions, but I'm realistic enough to realize where they are headed, and plan accordingly.

For the Mac desktop line up from the Studio on down to the entry model this notion of covering it all with iGPUs probably is a decent bet. The general PC market at the low end is drifting that way ( even AMD stuck a minimal iGPU on their mainstream desktop Ryzen series this time. When 2.5/3D packing gets more affordable that iGPU will likely grow. ). The dGPUs in the laptops in 2-3 generation on the PC side probably will shrink substantially. So Apple is ahead of the curve there.

But in the upper middle - to - high range GPUs they aren't ( CXL 2.0/3.0, more memory , bigger caches , and direct storage loads are going larger dGPUs in the game longer) . And the likely won't be as long as saving money by applying their laptop iGPU solution to the 'bigger problems'.

 

[deconstruct60]

When you look at it in detail, dGPUs only redeeming point is lower price. Which is achieved by sacrificing power efficiency and performance. And then there is the issue with interconnect which again makes things more complicated….

Not the only one. dGPUs scale out. iGPUs do not. There is a trade-off if set the Perf/Watt threshold too high. You basically cut yourself off from reaching certain performance levels to 'save' the ratio. If 'buying' power savings by piling on huge distance limitations (e.g., cannot be more than 3 inches away) then painting yourself into a performance corner.

 

[deconstruct60]

Is desktop that much of a niche? I mean, it makes sense that Apple focuses on laptops first, but it would also be nice if their desktops were competitive. Vertical scaling via a larger clock range sounds like a straightforward way to achieve this. In other words, what I am proposing here is to make a mobile-first product but also try to achieve good overclockability for desktop use.

Overclocking LPDDR RAM ? The extreme "Mobile first" position is also anchoring the components off the die as well. Apple can't extreme Turbo clock themselves out the corner they painted themselves into. They can be 'happy' in the corner, but they are in a corner.

 

[leman]

Not the only one. dGPUs scale out. iGPUs do not. There is a trade-off if set the Perf/Watt threshold too high. You basically cut yourself off from reaching certain performance levels to 'save' the ratio. If 'buying' power savings by piling on huge distance limitations (e.g., cannot be more than 3 inches away) then painting yourself into a performance corner.

Overclocking LPDDR RAM ? The extreme "Mobile first" position is also anchoring the components off the die as well. Apple can't extreme Turbo clock themselves out the corner they painted themselves into. They can be 'happy' in the corner, but they are in a corner.

Can you elaborate a bit more on this? Why would an iGPU be incapable of hitting higher frequencies? And why is the memory interface relevant, I thought that the internal clock and the memory clock are decoupled? Of course, as the clock increases the memory bandwidth might be not sufficient to feed the device anymore, but that can be solved in other ways.

 

[dandeco]

From what I've heard (and seen for myself), the integrated graphics using unified memory on Apple Silicon chips performs a lot better than the integrated graphics on Intel Macs (i.e. Intel HD/Ultra HD/Iris Pro graphics, or the integrated NVIDIA GeForce graphics seen on 2008-10 13" MacBooks (including Air and Pro). This is why it often comes close to performance akin to a powerful NVIDIA or AMD dedicated graphics card, but uses far less energy. (I mean, I was blown away by how my M1 MacBook Air can render/export a video project in just one third of the time it takes my 2012 quad-core i7 Mac Mini with 16 GB of system RAM and Intel HD Graphics 4000 chip, no matter what video-editing software I use.)

 

[deconstruct60]

I've seen this question asked multiple times, so I'll just respond with the same answer I've given previously.

I was curious about the question of expandability, specifically in regards to the Apple Silicon Mac Pro, so I decided to ask someone who actually understands CPU architecture and design. Former Opteron architect, Cliff Maier, who knows the engineers at Apple from his time at AMD and Exponential, and talks with his old colleagues regularly, had this to say. When I asked him about the Apple Silicon Mac Pro he replied with the following:



So, he believes a 1% chance of DIMMs, and 33% chance of discrete GPU but still on-package, just not integrated in the SoC

Same copy and paste job which doesn't represent what Maier was saying. Covered this before. He isn't talking about a discrete GPU. He is talking about GPU focused chiplet in a Apple SoC package. A GPU chiplet die doesn't get you a dGPU. What is talking about is an even bigger iGPU.

https://forums.macrumors.com/threads/october-revelations.2357700/post-31445657

Apple doing a dGPU doesn't make much sense economically or line up with they movements on software (ignoring dGPUs in macOS on AS ) or package configurations (relatively paltry PCI-e lane provisioning) . It is understandable why they don't want to do it themselves, but to actively block the discrete GPGPU from the Mac Pro workload space is dubious.

 

[deconstruct60]

Can you elaborate a bit more on this? Why would an iGPU be incapable of hitting higher frequencies? And why is the memory interface relevant, I thought that the internal clock and the memory clock are decoupled?

At some point you have to feed data to your cores for them to get pragmatic work done. You can arm wave about how the L2 cache and TBDR tile memory is going to hold all the data local, but at some point the faster you do the work the faster you run out of local data.

Cranking the clocks is why Nvidia's 4090 is in the power mess it is in. Apple won't magically avoid that if they follow the same path.

Of course, as the clock increases the memory bandwidth might be not sufficient to feed the device anymore, but that can be solved in other ways.

Those solution have limits. Cache hit rates drop at some point as try to grab more data. Apple is pretty close to the limit of "going wider" being a viable solution. There is only so much edge space on the die. They are already in the zone of trade-off general PCI-e I/O lane capacity for memory lanes.

The 'solved in other ways' is already being heavily leaned on to cover the LPDDR5 versus GDDR6 deficit.

 

[leman]

At some point you have to feed data to your cores for them to get pragmatic work done. You can arm wave about how the L2 cache and TBDR tile memory is going to hold all the data local, but at some point the faster you do the work the faster you run out of local data.

Cranking the clocks is why Nvidia's 4090 is in the power mess it is in. Apple won't magically avoid that if they follow the same path.


Those solution have limits. Cache hit rates drop at some point as try to grab more data. Apple is pretty close to the limit of "going wider" being a viable solution. There is only so much edge space on the die. They are already in the zone of trade-off general PCI-e I/O lane capacity for memory lanes.

The 'solved in other ways' is already being heavily leaned on to cover the LPDDR5 versus GDDR6 deficit.

Ok, so then your point is that vertical scaling via GPU core clock will hit a memory bandwidth limitation and not that pursuing wider clock range itself would be impossible or challenging.

To be honest, I do not share your pessimism. Apple currently generally offers higher RAM bandwidth per unit of GPU compute throughput than existing dGPUs (e.g. 40GBs per TFLOP for M1 Pro/Max/Ultra, ~25GBs per TFLOP for 3090, ~10GBs per TFLOP for 4090 etc.) so they still seem to have some headroom here (sure, some of this bandwidth is consumed by the CPU but that's just going to be some crumbles if you are going for a massive GPU workload). And they can still get more out of LPDDR5 by using the 8533 MT/s variant. For example a hypothetical Ultra based on the 5-core GPU cluster layout would have 10240 compute cores and memory bandwidth comparable to the RTX 4090. If Nvidia believes that 1TB/s is enough for 16k shader cores at 2.2Ghz, who am I to argue otherwise? The same hypothetical Ultra at, say, 2.2Ghz GPU clock could deliver ~45TFLOPs of sustained FP32 performance and would probably still consume under 200W...

 

[iPadified]

As some have been saying, decoupling the GPU cores from the CPU cores (same die approach) and pairing them via ultra fusion may work out better.

Sure, but it will not be cheap. MBP M1 Max, Studio Ultra all eats into the Mac Pro segment or do you expect custom configs CPU:GPU ratio for MBP and the Studio as well? I doubt that. Economy of scale.

So my point is, I doubt Apple wants to cater for this tiny market unless the company gain somewhere else such as marketing. If Apple would commit to 3D work (which they should have done like 2 decades ago) they would enter a new market and then everything is possible. That market is however very cost competitive which usually does not fit Apple.

NVIDIA gained a foothold on compute market years ago, not because it was better with GPU compute but because it was cheaper than CPU compute (Xeons). Intel practically priced themselves out of the workstation compute market and there is a risk that Apple will do the same. This has nothing to do with abilities to deliver but the will to deliver.

 

[sunny5]

People need to get over the idea that Apple will spend hundreds of millions in R&D to create additional custom SoCs and discrete GPUs for the Mac Pro. It's not going to happen.

If it does happen, it's because Apple will launch an Apple Silicon Cloud where anyone can rent a server powered by Apple Silicon.

Why not just use multiple M1,2 Max chips just like they did with M1 Ultra with two M1 Max?

 

[Boil]

Sure, but it will not be cheap. MBP M1 Max, Studio Ultra all eats into the Mac Pro segment or do you expect custom configs CPU:GPU ratio for MBP and the Studio as well? I doubt that. Economy of scale.

Apple can create a GPU-specific SoC, pair it with a Mn Max SoC; now they have skewed the ratio of cores to favor the GPU side of things...

Need more CPU cores than GPU, then go for a dual (Ultra) or quad (Extreme) Mn Max config...

Mn Max SoCs stay the same, still used in MBP laptops & Studio desktops; it is just the Mn Ultra Studios & the Mac Pros that have the option for a higher GPU core count, at the expense of a lower CPU core count...

So my point is, I doubt Apple wants to cater for this tiny market unless the company gain somewhere else such as marketing. If Apple would commit to 3D work (which they should have done like 2 decades ago) they would enter a new market and then everything is possible. That market is however very cost competitive which usually does not fit Apple.

I believe Apple is putting all the work into Blender to show what ASi/Metal is capable of and to show software developers how to properly implement ASi/Metal in their apps...?

 

[singhs.apps]

or do you expect custom configs CPU:GPU ratio for MBP and the Studio as well? I doubt that. Economy of scale.

Custom configs already exist in this economy of scale paradigm.

Else you wouldn’t get 8/14/16/24/32 GPU cores options whilst the cpu count remains a choice between just two - 8/10 cores.

In other words, Apple is offering compute differentiation revolving around GPU cores, not CPU cores.
De coupling the GPU cores from the SOC and having them connect via ultrafusion may allow Apple to offer even higher core count GPUs (10/12 core CPU + 64 core GPU) rather than being stuck by the physical dimensions of the SOC.

It may not happen in the next few iterations of the AS, but at some point moving out the GPU/accelerators may allow Apple room to grow in segments that need not be constrained.

I have zero idea about hardware design so take the above with a pinch of salt.

 

[Colstan]

The pot calling kettle black.... same boat with the "Apple is doing dGPU" thing with the post #26 in this thread.

Yes, we have covered this before, and I responded to you in this thread for elaboration. I tried to get a response in a friendly manner, but you didn't reply.

Mac Pro 2019 --- power supply 1400W

Yes, I know, I've got one in the Mac Pro on my computer desk sitting next to me. I've known this since it was released. My point was that the the vast majority of dGPUs require substantial power and the Mac Pro is a niche product that I don't think Apple is going to spend resources catering to.

Same copy and paste job which doesn't represent what Maier was saying. Covered this before. He isn't talking about a discrete GPU. He is talking about GPU focused chiplet in a Apple SoC package. A GPU chiplet die doesn't get you a dGPU. What is talking about is an even bigger iGPU.

Yes, I know, that has been my understanding the entire time, from the time I first read it. We are not in disagreement on this, we have never been, thus my confusion. That's why I asked for clarification, but you seem to be hung up on the way I am communicating it. At this point we are talking past each other even though I agree with what you are saying here.

Ruminating on it, I think what you are missing is context. When I had this conversation with Maier, we were discussing how Apple may implement the Mac Pro GPU, in the fashion you describe right here. Somewhat separately, we discussed the chances of a third-party GPU, in a related fashion. So, in my mind they are linked because I already went through this, but you are seeing my interpretation of bits and pieces, without the entire, lengthy discussion.

As I said in the link above, I apologize for any confusion I may have caused. This is simply a communication issue between the two of us, but nothing more. I don't know what else to tell you.

 

[iPadified]

Custom configs already exist in this economy of scale paradigm.

Else you wouldn’t get 8/14/16/24/32 GPU cores options whilst the cpu count remains a choice between just two - 8/10 cores.

In other words, Apple is offering compute differentiation revolving around GPU cores, not CPU cores.
De coupling the GPU cores from the SOC and having them connect via ultrafusion may allow Apple to offer even higher core count GPUs (10/12 core CPU + 64 core GPU) rather than being stuck by the physical dimensions of the SOC.

It may not happen in the next few iterations of the AS, but at some point moving out the GPU/accelerators may allow Apple room to grow in segments that need not be constrained.

I have zero idea about hardware design so take the above with a pinch of salt.

Current scaling revolves around a few standard chips, not adding a separate GPU board to a Max and that makes a huge difference. Although a logical path, a separate configurable GPU board connected to a Max chip will be akin to adding an Afterburner functionality. This translates to very expensive because very few need this amount of GPU performance.

Sorry for being pessimistic but I cannot see the business sense in competing with AMD/NVIDIA because Apple silicon and Metal is not really what most software is written for. Apple has a long uphill struggle to move past video/photography/music markets.

 

[mode11]

as opposed to merely making it a 2x Ultra. [Though note that, by definition, even the latter still necessarily entails a different, larger SoC with more memory chips.]

Might it be possible that instead of UltraFusion directly linking 2 SoCs, 4 SoCs could be linked via UF to some kind of central hub / switch? Bit like a PCIe PLX switch. Or the way you connect two PCs together with an Ethernet crossover cable, but to connect more you use a switch.

The problem is that this ancient way of designing leaves modern Apple in a puff of dust. For the end user, what matters is how quickly the job is done.

This. Plus, let's not forget that whatever the current performance / Watt advantages, the primary motivation for moving Macs to Apple Silicon is to lower costs / increase profits. Not just because they make the chips themselves, but because porting macOS to ARM means it can increasingly share development with their main operating system, iOS.

It’s the same thing really, if Nvidia can build a huge GPU and give it a 2+ghz clock why can’t Apple?

Because Nvidia makes GPUs with massive heatsinks, whereas Apple makes energy-sipping SoCs. Can't see Apple putting an RTX3090's (or even a 3060's) worth of firepower on the same SoC die as a load of CPU cores, neural engines and whatever else. Not unless the die will be the size of a beer mat and consume 500W.