Apple GPUs speculation: TBDR, RT and the benefit of wide ALUs

[awesomedeluxe]

The price for a 300mm N5 wafer is allegedly $16,988.

That brings it down to a mere $130 per chip at those yields.

Cheap! The i5-1038NG7 has a "recommended customer price" of $320. I'm sure Apple gets some kind of volume discount, but I doubt it cuts the price by more than half!

I figure they can probably salvage a big chunk the bad chips too. These bigger dies are more defect-prone, but at least they're binnable, while a bad A14 just has to be thrown out. Too bad TSMC missed their target for ramping up N5P--the M99 would've been a great candidate for low volume production to iron out any issues with the process ahead of the A15.

Yeah, I see what you mean. It's an issue of initial heat transfer more than overall heat dissipation. No idea what modern vapor chambers are capable of...

Ok, I'm pretty confident now that if Apple leaves the two big fans they've got in there things should be fine. A design like this would work:

That's a Matebook 14. Same thickness as a MBP16 and uses a single APU (Ryzen 4800H) with two heat pipes pulling heat away to two separate fans. Sustains at 37W over 1.57cm2--23.57W/cm2.

Most people I've talked to think it wouldn't be a problem for heat pipes to pull heat away from the APU quickly enough as long as the fans are large enough and powerful enough to shed the heat. And based on its size (it's 1.1 MBP13s or .8 MBP16s by volume), the limit on the Matebook 14 probably has more to do with total heat dissipation than heat/area.

 

[theorist9]

Even if it costs Apple $500 to manufacture one MacBook Pro 16” SoC, it’s still significantly cheaper than buying chips from Intel and AMD. Not to mention that they are also saving costs on the main board as well. They could live with worse yields and still make a good profit over the Intel Macs.

Do you know what they currently pay Intel and AMD for the CPU's & GPU's in that machine? I wasn't able to find that info. (or even informed speculation on that, other than it's much less than retail [obviously]).

As you know, part of Apple's strategy in having its own in-house processors is to save money. This article estimates the M1 chip is costing Apple $100/unit from TSMC (that doesn't include, of course, Apple's extensive R&D costs): https://wccftech.com/apple-custom-mac-cpu-cost-node/ By comparison to that, at least, $500 is a lot.

What kind of chip did you have in mind for that $500? Are you thinking about a maxed-out design for the MBP with, say, 16+4 CPU/32 GPU?

 

[leman]

Do you know what they currently pay Intel and AMD for the CPU's & GPU's in that machine? I wasn't able to find that info. (or even informed speculation on that, other than it's much less than retail [obviously]).

I haven't seen any data on that either, but I'd assume that CPU+GPU+RAM for the 16" is going to be close to $800+.

As you know, part of Apple's strategy in having its own in-house processors is to save money. This article estimates the M1 chip is costing Apple $100/unit from TSMC (that doesn't include, of course, Apple's extensive R&D costs): https://wccftech.com/apple-custom-mac-cpu-cost-node/

Isn't $100 a bit much? Maybe it's $100 for the entire package, including RAM etc.?

What kind of chip did you have in mind for that $500? Are you thinking about a maxed-out design for the MBP with, say, 16+4 CPU/32 GPU?

I was thinking about the 16" part, which I would speculate to have a 8+4 (or 12+4) CPU cores and 32 GPU cores (which they could bin into a 16-core and 32-core part).

 

[Pressure]

I haven't seen any data on that either, but I'd assume that CPU+GPU+RAM for the 16" is going to be close to $800+.



Isn't $100 a bit much? Maybe it's $100 for the entire package, including RAM etc.?



I was thinking about the 16" part, which I would speculate to have a 8+4 (or 12+4) CPU cores and 32 GPU cores (which they could bin into a 16-core and 32-core part).

I calculated around $71 per die (119mm2) or 241 good dies per wafer.

 

[theorist9]

I haven't seen any data on that either, but I'd assume that CPU+GPU+RAM for the 16" is going to be close to $800+.

If you figure materials costs on a $2700 MBP are 60% of retail (again, just guessing. b/c these figures are hard to come by) then, at $800, the CPU+GPU+RAM would represent half the BOM. Is that reasonable? Maybe it is--I don't know myself.

 

[Unregistered 4U]

However they do it, I hope Apple keeps some GPU capability in the "high-end" (32-Pcore/4-Ecore SoC w/32-core GPU & 32-core Neural Engine), allowing folks who need the PCIe slots (of which there will be fewer in the forthcoming smaller Mac Pro), meaning the audio guys mainly, to not need to fill one of the remaining PCIe slots with a GPU...

So, I noticed that the current storage for the M1BP is “Apple Fiber”. Has anyone seen any specifications on that? Seeing as how the System Report indicates no PCI ports or devices when there’s nothing connected, does that mean that everything’s on Apple Fiber? Would any future GPU’s also have to be on Apple Fiber (non-industry standard, but, since we’re already separated from the CPU, why not go the rest of the way?)

 

[awesomedeluxe]

I haven't seen any data on that either, but I'd assume that CPU+GPU+RAM for the 16" is going to be close to $800+.



Isn't $100 a bit much? Maybe it's $100 for the entire package, including RAM etc.?



I was thinking about the 16" part, which I would speculate to have a 8+4 (or 12+4) CPU cores and 32 GPU cores (which they could bin into a 16-core and 32-core part).

If you figure materials costs on a $2700 MBP are 60% of retail (again, just guessing. b/c these figures are hard to come by) then, at $800, the CPU+GPU+RAM would represent half the BOM. Is that reasonable? Maybe it is--I don't know myself.

I'd guess less. Even though the Radeon parts are semicustom (in the sense that AMD was sort of making these GPUs to Apple's specifications before trying to resell them to other manufacturers) Apple was still getting them for a song and that's both because 1. AMD was desperate for business as recently as two years ago and 2. there was not a lot of demand for AMD's graphics parts from anyone else

I also think it's more likely than not that Apple was able to exert pricing pressure on Intel as one of its largest customers. I don't see Apple paying $400 a pop for i9-9880Hs. And volume pricing for DDR4 is much lower than retail.

Altogether I think we are looking at around $600. This is still a very high ceiling to get under. But if their M1X SoCs we costing $500, I'd consider that a failure.

So, I noticed that the current storage for the M1BP is “Apple Fiber”. Has anyone seen any specifications on that? Seeing as how the System Report indicates no PCI ports or devices when there’s nothing connected, does that mean that everything’s on Apple Fiber? Would any future GPU’s also have to be on Apple Fiber (non-industry standard, but, since we’re already separated from the CPU, why not go the rest of the way?)

Do you mean Apple Fabric, maybe? No one knows exactly what it is but we aren't inclined to think it's anything special. Like AMD Infinity Fabric - it's probably just standard interconnect technology with a cool name.

 

[Unregistered 4U]

Do you mean Apple Fabric, maybe?

LOL! Yes, Apple FABRIC Such a generic term, a Google search isn’t going to bring up anything company related for awhile!

 

[jeanlain]

I thought this could be relevant:

The findings of the process documented in the free software disassembler confirm a number of traits of the GPU:

One, this is a scalar architecture. Unlike some GPUs that are scalar for 32-bits but vectorized for 16-bits, the M1’s GPU is scalar at all bit sizes. Yet Metal optimization resources imply 16-bit arithmetic should be significantly faster, in addition to a reduction of register usage leading to higher thread count (occupancy). This suggests the hardware is superscalar, with more 16-bit ALUs than 32-bit ALUs, allowing the part to benefit from low-precision graphics shaders much more than competing chips can, while removing a great deal of complexity from the compiler.

Dissecting the Apple M1 GPU, part I

rosenzweig.io

 

[leman]

I thought this could be relevant:

Dissecting the Apple M1 GPU, part I

rosenzweig.io

ALU throughout benchmarks don’t support this assumption. Max throughout of FP32, FP16 and mix thereof are identical on M1. On A14, FP32 is half the throughput rate of FP16. Since the GPU cores seem identical otherwise, the obvious explanation is that that the same ALUs are used for both precision’s, but A14 slows down FP32 execution, most likely to save power.

FP16 precision will of course use less local storage and less memory bandwidth, but that has been a common optimization for all GPUs anyway. So even if M1 can’t process them faster, there are good reasons to use FP16 where it makes sense.

 

[diamond.g]

@leman do you know of anyone interested in porting Quake 2 RT (Q2VKPT) which uses Vulcan to Mac using MoltenVK. I am curious if MVK can translate VK_KHR_ray_tracing_pipeline to the Apple equivalent.

 

[leman]

@leman do you know of anyone interested in porting Quake 2 RT (Q2VKPT) which uses Vulcan to Mac using MoltenVK. I am curious if MVK can translate VK_KHR_ray_tracing_pipeline to the Apple equivalent.

Sorry, I don’t know anyone like that. I don’t know if MoltenVK people are working on RT support, at least I can’t see anything in the issue tracker. I wouldn’t hold my breath though... a lot of things need to happen before supporting RT in MoltenVK makes practical sense, not the least of which is having hardware RT in Apple GPUs.

 

[diamond.g]

Sorry, I don’t know anyone like that. I don’t know if MoltenVK people are working on RT support, at least I can’t see anything in the issue tracker. I wouldn’t hold my breath though... a lot of things need to happen before supporting RT in MoltenVK makes practical sense, not the least of which is having hardware RT in Apple GPUs.

Any reason why Apple can't enable it on shaders (like Nvidia did for the 10 Series cards)?

 

[leman]

Any reason why Apple can't enable it on shaders (like Nvidia did for the 10 Series cards)?

Ah, I was not aware that they also support it on GPUs without RT acceleration now. Cool! Yes, that’s exactly what Apple is doing with Metal ray tracing - it’s implemented using compute GPU functionality and runs on all Macs AFAIK, but the problem is performance. You can use it for some limited stuff or for offline rendering, but its not fast enough to support real-time large-scale rendering in modern games. Something like Quake2 might work, no idea, but MoltenVK developers are focusing on features that have more practical use.

Quake2 could be probably ported to use Metal directly with relative ease, maybe someone will be interested of doing it as a little side project.

 

[diamond.g]

Ah, I was not aware that they also support it on GPUs without RT acceleration now. Cool! Yes, that’s exactly what Apple is doing with Metal ray tracing - it’s implemented using compute GPU functionality and runs on all Macs AFAIK, but the problem is performance. You can use it for some limited stuff or for offline rendering, but its not fast enough to support real-time large-scale rendering in modern games. Something like Quake2 might work, no idea, but MoltenVK developers are focusing on features that have more practical use.

Quake2 could be probably ported to use Metal directly with relative ease, maybe someone will be interested of doing it as a little side project.

That would be interesting. The base game is written using OpenGL. The RT version using Vulkan is fully path traced so there is no rasterization going on (from my understanding).
I actually wonder if the Neural Engine can be leveraged for RT acceleration.

 

[leman]

I actually wonder if the Neural Engine can be leveraged for RT acceleration.

I dint see how. Neural Engine is most likely a specialized low precision matrix multiplication unit and you can’t really do ray tracing in those. Frankly, I haven’t found much information on what Neural Engine actually does, it appears to be a highly specialized coprocessor at any rate. Apple seems to use AMX units for general-purpose ML and things like denoising (which is crucial for RT) are actually done on the GPU compute shaders. This suggest that Neural Engine is either unable to do these tasks or there is no benefit of using it to do them.

 

[diamond.g]

I dint see how. Neural Engine is most likely a specialized low precision matrix multiplication unit and you can’t really do ray tracing in those. Frankly, I haven’t found much information on what Neural Engine actually does, it appears to be a highly specialized coprocessor at any rate. Apple seems to use AMX units for general-purpose ML and things like denoising (which is crucial for RT) are actually done on the GPU compute shaders. This suggest that Neural Engine is either unable to do these tasks or there is no benefit of using it to do them.

Thanks, that is informative.

 

[jeanlain]

Sorry, I don’t know anyone like that. I don’t know if MoltenVK people are working on RT support, at least I can’t see anything in the issue tracker. I wouldn’t hold my breath though... a lot of things need to happen before supporting RT in MoltenVK makes practical sense, not the least of which is having hardware RT in Apple GPUs.

macOS 11.4 has drivers for GPUs with dedicated RT hardware (radeon 6000 series). I wonder if Metal takes advantage of that hardware.
More generally, would RT hardware be leveraged "automagically" by the driver, or will it require different APIs from shader-based RT?

 

[leman]

macOS 11.4 has drivers for GPUs with dedicated RT hardware (radeon 6000 series). I wonder if Metal takes advantage of that hardware.
More generally, would RT hardware be leveraged "automagically" by the driver, or will it require different APIs from shader-based RT?

The API would be the same (Metal RT API is very similar to DX12 RT for example). Actually, Metal RT API is clearly designed with hardware RT support in mind (RT specific things are neatly abstracted away and just waiting for hardware intersectors and acceleration structure builders).

Great question about the Navi 2! I’d also love to know. Hopefully someone with one of these new cards can do a quick test...

 

[jeanlain]

Great question about the Navi 2! I’d also love to know. Hopefully someone with one of these new cards can do a quick test...

You can always ask posters from this thread. The 6900XT performs well in octane X, but not much higher than expected from its TFLOPS.
I don't know if octane uses Metal RT APIs or some custom shader suff from OTOY.

BTW octane results from the M1 are disappointing. It's much slower than the 5300M.

 

[diamond.g]

The API would be the same (Metal RT API is very similar to DX12 RT for example). Actually, Metal RT API is clearly designed with hardware RT support in mind (RT specific things are neatly abstracted away and just waiting for hardware intersectors and acceleration structure builders).

Great question about the Navi 2! I’d also love to know. Hopefully someone with one of these new cards can do a quick test...

I guess I could ask in the other thread, but World of Warcraft supports RT (at least on Windows) so it should be an easy check to see if the option is available with the 6900XT plugged in on macOS.

 

[Bodhitree]

Very informative thread, thanks all. This may be a bit controversial, but it looks to me like RT is a feature whose time has not yet come, if you have to pair it with a technique like DLSS. Side-by-side in-engine comparisons I have seen showed that the visual upgrade from standard shading techniques isn‘t that visible to most people, if you knew what you were looking for you could see it, but if you were focussed on just absorbing the view it wouldn’t immediately strike you. Most serious AAA game engines are already pretty good.

There are an awful lot of games that are not perfectly photorealistic but provide a very good overall experience, and it seems to me that the overall experience is where Apple want to focus their gaming effort. There are plenty of games out there that prove that cutting edge rendering technology is not required. Take Animal Crossing, it is played on the Nintendo Switch and has sold 31 million copies in its first year. There is no reason something like that couldn’t run on the A14, let alone the M1.

 

[Unregistered 4U]

There are an awful lot of games that are not perfectly photorealistic but provide a very good overall experience, and it seems to me that the overall experience is where Apple want to focus their gaming effort. There are plenty of games out there that prove that cutting edge rendering technology is not required. Take Animal Crossing, it is played on the Nintendo Switch and has sold 31 million copies in its first year. There is no reason something like that couldn’t run on the A14, let alone the M1.

Ray tracing where Apple’s concerned might be about providing for a more realistic AR experience. Being able to detect and apply appropriate drop shadows, reflected colors, etc.

 

[diamond.g]

Very informative thread, thanks all. This may be a bit controversial, but it looks to me like RT is a feature whose time has not yet come, if you have to pair it with a technique like DLSS. Side-by-side in-engine comparisons I have seen showed that the visual upgrade from standard shading techniques isn‘t that visible to most people, if you knew what you were looking for you could see it, but if you were focussed on just absorbing the view it wouldn’t immediately strike you. Most serious AAA game engines are already pretty good.

There are an awful lot of games that are not perfectly photorealistic but provide a very good overall experience, and it seems to me that the overall experience is where Apple want to focus their gaming effort. There are plenty of games out there that prove that cutting edge rendering technology is not required. Take Animal Crossing, it is played on the Nintendo Switch and has sold 31 million copies in its first year. There is no reason something like that couldn’t run on the A14, let alone the M1.

At least with the 30 series of cards DLSS is really useful to run games at 4K with RT. Games running 1080P or 1440P do get a boost, but for the most part games are already running over 60fps at those resolutions with RT anyways.

I guess I am curious really if Apple will whiff RT performance like AMD did for the first attempt, or will they be able to match nvidia’s efforts. Without DLSS of course, I don’t think Apple is going to bother making a DLSS equivalent.

 

[EntropyQ3]

I guess I am curious really if Apple will whiff RT performance like AMD did for the first attempt, or will they be able to match nvidia’s efforts. Without DLSS of course, I don’t think Apple is going to bother making a DLSS equivalent.

For better or worse, Apple is taking a mobile -> desktop technology approach.
The main advantage for users is ergonomics - sleek, quiet, cool computers that still pack a punch, particularly because they have dedicated functional blocks that deal with common ”heavy lifting” tasks, and a focus on efficient use of resources.
I would guess that a consequence of that will be that technologies that don’t make a lot of sense on mobile for efficiency reasons will have low priority. Will adding dedicated hardware blocks to do some shadowing and reflection calculations in simulated 3D environments make the cut, when it can already be achieved without them? I’m not so sure if that’s a good idea.

Quality upscaling is a good thing in general. Apple devices have a sufficiently high output resolution that individual pixels are hard or impossible to discern. Thus it’s often a good idea to render 3D at a lower resolution and upscale it to device resolution, as it saves a lot of resources and improves performance. The question is how to go about it.
DLSS is a method that requires support in the rendering pipeline. It is introduced before the post-processing stages that are done at full output resolution. It’s quite costly but yields very good results for the applications that are coded to take advantage of it. But it is not necessarily the best method for Apple to promote. For instance, they have no reason to chase the best possible pixel level results, because small differences from the ideal in those pixels aren’t visible anyway. Apple may be better served by upscaling that is more rendering pipeline agnostic and easier to implement as a last step before output. It can still be pretty damn good, and utilize the NPU for acceleration!

Nvidia is selling DLSS hard because it is a checkmark advantage over AMD in the gaming market, but it is no panacea, and the requirements it has for implementation may not fit Apples eco-system particularly well.