That’s certainly fair. I was referring mostly to the stock valuation bubble looking basically the same, since that’s what the guy was talking about with respect to NVDA market cap.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Mac Pro should use NVIDIA cards
- Thread starter Zest28
- Start date
- Sort by reaction score
I would describe LLMs as databases, but instead of storing factual data or descriptions they store the statistical makeup of human-produced language itself. I think what it tells us in the end how much structure there actually is in the signal alone, without even taking meaning or intent into account. It’s a topic that’s very interesting to me since it is fairly close to what I did for my PhD (although I approached it from a different angle). But yeah, it has very little to do with “intelligence” as we understand it. It’s a stochastic regurgitator that has been trained on everything humans have ever produced. However, pair this machine with an actual database, a symbolic reasoning engine, and an evaluation engine that is capable to adopt different virtual persona and you’ll get very close to real intelligence.
I agree with you that Nvidia is likely a bubble since their hardware is not very power efficient and also not particularly reliable. There are a lot of companies right now working on ML-focused solutions, and I think there is a decent chance that Nvidia will lose their market dominance relatively soon.
I don't think Nvidia has anything to worry about, as long as everyone relies on TSMC and there is a shortage of high-end fab capacity. Maybe Apple could be a serious competitor, if they decided to use some of their reserved capacity for AI chips. But apart from Apple, Nvidia is capable of outbidding everyone else, and they still can't make enough chips to meet the demand.
The lack of power efficiency is a consequence of that. As long as the availability of chips is a bigger constraint than power or cooling, efficiency means performance per die area.
On the Mac Pro? Maybe not directly, i.e., when it comes to their power bills. But MP customers that have the machine next to their desks do care about the consequence of poor power efficiency, which is high thermal output, as that in turn means a hotter and noisier machine.
Actually, I think I pretty much do. [For those other than me and Dave: We were discussing whether Apple blocks 3rd-party slotted SSD modules; I contend they do.]
Just to verify, I contacted OWC about this. As I mentioned before, they're one of the best-known experts on producing 3rd-party upgrades for Macs. And they confirmed they looked into this and found that, as of this time, it was not technically possible, because Apple doesn't allow it.
Here's the transcript of the chat (I edited out my name and the name of the OWC rep, but otherwise the content is unchanged). I asked if the reason OWC doesn't sell 3rd-party slotted SSD modules, for the Studio or MP, is because this is blocked by Apple. He said yes:
OWC: Thanks for choosing OWC for your technical needs. I would be more than happy to assist! Just to begin with correct they made it non user changeable so that is not something we can currently offer. If we eventually find a solution for that we will post about it but as of current is unlikely
Me: ....and just to clarify, when you wrote "non user changeable", you meant non-user changeable with 3rd party solutions, right? Since these are user-changeable on the MP when you buy the part from Apple.
OWC: Correct
In addition, if you take a closer look at that SSD upgrade thread you linked, you'll see they were only able to do storage upgrades on the Studio by soldering higher-capacity NAND onto existing Apple-branded slotted SSD modules. No one is able to make 3rd-party slotted SSD modules, because Apple blocks their operation. The only workaround is to use soldering to upgrade Apple-branded modules, and the only way you can get those is by pulling them from a Studio, or possibly through Apple's self-repair program.
That's not going to block 100% of customers from upgrading them, but it will block 99.[many 9's]% of them, making it a block for all intents and purposes.
Here's a dedicated thread on Mac Studio storage upgrades:
Upgrading Mac Studio storage
Hi, just tested yesterday an upgrade of a plain basic MacStudio M1 Max (2022) from 512GB to 4TB. and it works : take off the 512 GB NAND insert in the right order the two 2TB NANDs inside the slots restaure the MacStudio bridgeOS with another Mac and AppleConfigurator et voilà ...
forums.macrumors.com
Last edited:
I think it is important that we distinguish between energy consumption and power efficiency in these conversations.
Power efficiency is one of the most important criteria for an enthusiast user, because higher power efficiency at the same power usage = higher performance. The problem of Mac Pro is that it has a considerably lower power ceiling, limiting performance. If Apple would supplied hardware that targets the same ~ 800-1000W bracket at the 2019 MP, the performance would be off the charts.
Ok maybe I'm reading this wrong, but that thread actually confirms what I was saying earlier: there is no "software block" (in fact such a thing would be impossible given the nature of the upgrade), it's an economic block on these slotted SSDs by making it difficult to reverse engineer and get the necessary components. However even then the OP is in fact starting to reverse engineer the ability to add NANDs and sell his services to allow upgrades at lower than the price Apple sells upgrades on the Pro for:Actually, I think I pretty much do. [For those other than me and Dave: We were discussing whether Apple blocks 3rd-party slotted SSD modules; I contend they do.]
Just to verify, I contacted OWC about this. As I mentioned before, they're one of the best-known experts on producing 3rd-party upgrades for Macs. And they confirmed they looked into this and found that, as of this time, it was not technically possible, because Apple doesn't allow it.
Here's the transcript of the chat (I edited out my name and the name of the OWC rep, but otherwise the content is unchanged). I asked if the reason OWC doesn't sell 3rd-party slotted SSD modules, for the Studio or MP, is because this is blocked by Apple. He said yes:
OWC: Thanks for choosing OWC for your technical needs. I would be more than happy to assist! Just to begin with correct they made it non user changeable so that is not something we can currently offer. If we eventually find a solution for that we will post about it but as of current is unlikely
Me: ....and just to clarify, when you wrote "non user changeable", you meant non-user changeable with 3rd party solutions, right? Since these are user-changeable on the MP when you buy the part from Apple.
OWC: Correct
In addition, if you take a closer look at that SSD upgrade thread you linked, you'll see they were only able to do storage upgrades on the Studio by soldering higher-capacity NAND onto existing Apple-branded slotted SSD modules. No one is able to make 3rd-party slotted SSD modules, because Apple blocks their operation. The only workaround is to use soldering to upgrade Apple-branded modules, and the only way you can get those is by pulling them from a Studio, or possibly through Apple's self-repair program.
That's not going to block 100% of customers from upgrading them, but it will block 99.[many 9's]% of them, making it a block for all intents and purposes.
Here's a dedicated thread on Mac Studio storage upgrades:
Upgrading Mac Studio storage
Hi, just tested yesterday an upgrade of a plain basic MacStudio M1 Max (2022) from 512GB to 4TB. and it works : take off the 512 GB NAND insert in the right order the two 2TB NANDs inside the slots restaure the MacStudio bridgeOS with another Mac and AppleConfigurator et voilà ...
https://forums.macrumors.com/thread...ge.2370048/page-3?post=32921772#post-32921772
Sure in the OP he thought it wouldn't be possible. But now? He's doing it. In other words, it's just a matter of time. Unfortunately it's past 1 here and I'm posting exhausted so perhaps I'm not understanding but what I'm seeing, there's a 3rd party preparing to sell slotted SSD upgrades based on reverse engineering Apple's SSDs and we already know that there is a 3rd party Chinese market with the soldered SSDs.
Now relating that back to our larger conversation, you'll note that in my analogy I related Apple's soldered/slotted SSD to the their UMA memory which is definitely not user-upgradeable. So even if the SSDs were somehow completely blocked or we just accept that economic block is for all practical purposes as good as completely blocked, my analogy still holds. Which is why as interesting as this conversation is, it is mostly a tangent to that earlier topic. I still believe my earlier arguments with regards to that hold. Even more so with RAM, even Apple tried to offer DDR memory in specially designed modules that only worked from them, if the market is big enough, heck just out of sheer interest, someone will reverse engineer it. It's only a matter of time. Apple could slow it down, but can't stop it.
But since we're on that tangent of the slotted SSDs, I have to say, Apple's strategy here makes no sense. They offer 1st party upgrades on the one device where 3rd party upgrades via internal PCIe are just as easy and cheaper. And on the Studio it also has slotted SSD where again even though it isn't internal, it's a decent sized desktop and TB4 external storage is simply not as clunky as for say a laptop or even a mini (debatable). TB5 will only exacerbate that. I can only think of two rationales though I consider them both weak:
1) This is in part why the Mac Pro base price is so out of whack, i.e. part of why they want to get higher margins at the base price. Since there is an "expectation" of expandable storage they price the base model as though they aren't going to get as much money from people paying them for doing so. Because they aren't.
2) The slotted SSDs on the Pro and the Studio have to do with the Ultra chip. Even if SSD failure under warranty is rare, throwing an entire motherboard away for it when that motherboard has a very expensive Ultra chip is just not acceptable cost (why it's acceptable for a still quite expensive Max chip not in a Studio or any of the others even if they don't cost as much is a different question or maybe the same question but asked louder and with greater exasperation*)
Basically I don't think Apple's strategy here is very well thought out. It makes a little sense for their primary market: laptops where internal expansion is impossible and external storage expansion can be clunky depending on what you want to use it for (though again just adopt the slotted Mac studio system and if post-sale upgrades are "practically blocked" for that then you still protect margins and get to save on repair costs?*). But for desktops it is extremely questionable.
*the answer of course is that then 3rd party SSD slotted modules would no longer be economically blocked if every Mac sold had one instead of some being soldered. The economic incentive to reverse engineer the SSD slotted module, no matter how complicated, would be gargantuan. Of course, as far as I can tell it's happening anyway so ...
Last edited:
'database' is an exceedingly poor description here. Even in the early database era of IMS and CODASYSL you would be extremely hard pressed to find anyone refering to the results from a DB query as a 'hallucination'. And even less so in the relational era or non-relational implementations like Berkley DB or Adabas.
A 'repository' perhaps. But effectively this is more like a playing Jenga with a 'black box ball" of interdependent sticks that if pull on it the wrong way it unwinds ( 'hallucinates'. )
A 'database' has a higher dedication evolutionary growth to the truth and higher data integrity threshold then these 'neural net language approximations' ( which are substantially subject to same problems that humans have with their 'neural net' when it comes to truth and data integrity over the long term. )
in a RDMS you keep the foriegn key relations clean and in these system they are excessively dribbled on everything like ketchup.
For these LLM here is "meaning and intent' in telling folks what they want to hear. (and fill in their own predisposed blanks) and sprinkle some syntax sugar ( 'structure') and it goes a long way. That is while 'piled higher and deeper' gets more traction. And not devoid of 'meaning' because "translation" is a task being assigned to several of these models on a regular basis.
Therein lies a bigger chunk of the 'rub'. What is "intelligence" has shifted, evolved, etc. over time. There are variety of aspects to that common holistic characteristic that folks used to commonly just assign to humans.
For a long time the "Intelligence" test was whether the entity behind a curtain could carry on a modern length conversation or not. Or can compose 'nice' music. Or can paint a 'nice' picture.
Other animals ( and plants) learn.... do or don't want to apply intelligence to that? ( or is only a "humans looking at themselves" thing? ).
" xxxx engine .... evaluation engine" ... chuckle. This has been a long , long issue with realizing AI along the lines of the old far side cartoon
And then a miracle happens…
This has always been one of my favorite cartoons. You know when your sitting in a meeting and somone presents a problem and then somehow in the middle of the solution “a miracle happens”…
avionod.wordpress.com
Along the lines of the recent Google Gemini 'feature' that stuffs extra "stuff" into the query so that the 'right' answer pops out. And the old MIT Media Lab (paraphrased AC Clarke) saying of "Any sufficiently advanced technology is indistinguishable from a rigged demo"
This is a very specific case. And Vienna is just a damm large sample server, dont need even a beefy cpu for it, just memory, most composers I know that uses it, just runs it outside in an old PC with lots of RAM and feeds the main Mac machine, where the actual mix and edit session is made.
Why people in foruns tend to think any kind of info is valid to build an argument… gosh.
Mac Pro or Workstation is meant for specific uses. Since Mac uses is totally limited, its Apple fault by limiting what Mac can do which is 2D area. Too bad since Apple is not focusing on professional and high end after all.
The fact that you are saying this means you didn't understand my argument. I never argued it wasn't very specific. Indeed, I acknowledged precisely that. Rather, I was arguing that it wasn't pathological. And specific is entirely different from pathological.
More importantly, why are some people in these forums incapable of responding to arguments without taking a gratutious personal swipe? Why not just simply say why you disagree, but otherwise keep it civil and friendly?
How would you like it if I responded to your post by saying: "Why is it that people who clearly lack the intellectual capacity to understand my posts presume to criticize them?" Just because it's true in this case doesn't mean one should say it.
Yeah, I did swing back at you--but that's understandable; what's not understandable is taking the first swing at someone who's done nothing to you.
I suppose this happens in part because there's a subset of people who post on these forums simply because they lack the social skills to interact well in real life, which is is why you get this anti-social behavior.
Last edited:
A narrow range doesn't necessarily absolve it from being pathological.
In a substantive number of cases with super-mega footprint orchestral DAW set ups it is a 'compulsive' pathology in the mix. When drag 80-100% of every possible sample you have on your storage drive into memory ( and buy as many samples are possibly can from vendors ) , "just because might need it" is a pathology.
[ Vienna Big Bang Orchestral Bundle .
" ...
- Free storage space: 508.4 GB
BBO BUNDLE - Vienna Symphonic Library
The BIG BANG ORCHESTRA BUNDLE contains all the individual Big Bang Orchestra Collections from A to Z. Create large orchestrations in no time. Beef up your existing tracks with ensemble and solo instruments, and inject some inspiration into your composing process.
300GB would be approximately 60% of everything in that giant bundle.
]
Similar issue with "the disk caching mechanism in software package X is bad " so preload and stage maximum amount of stuff into RAM. Again "badly written software" is also a pathology.
I am betting that we could load a much larger polygon laden scene with massive textures a lot easier on my 128GB M3 than a RTX4090 which is limited to 24GB of VRAM. Even double the VRAM still leaves me 80GB of RAM for whatever else.
Now is the 4090 way more performant than the M3; well of course. But it also sucks up 100x or more wattage doing so.
I am actually more curious what nVidia is going to do for the 5000 series cards. Even more so since they have released their Blackwell server GPU iron that has 2 chipsets (cores) with a coherent memory of 192GB. Although the TDP is kind of insane at 1000W. If they can push whatever they are doing on those chips down to the HEDT line of GPUs, things are really going to get interesting.
Clueless about what everyone is talking about here regarding all the back end things and processes. I was surprised though that my 5 year old gaming PC with an i7 9700 and an RTX 2070 and 16 gigs of ram destroyed my M1 Max with 32 gigs of ram when it came to AI creation in Krita. Matching settings, the PC would render an AI scene significantly faster then the M1 Max. Krita has an option on Windows to use something specific for an NVIDEA gpu which I am guessing is taking all that hard work and zipping through it.
The RTX 2070 is nominally fairly similar in performance to M1 Max, but it also has dedicated ML compute features and software optimization plays a big role too. Apple has no hope competing with Nvidia on small-size ML problems until they introduce support for packed formats and more matrix compute.
Is there any indication that they are planning to do this in the near future—or, conversely, any indication that they are not?
I could see this being a must if you were doing this sort of thing, or anything really fully utilizes a GPU. I work in graphics and love my M1 Max running Indesign, Illustrator and Photoshop. Where it does lag is with 3D box viewing, checking a design and how it all goes together. I think in this case part of it might be the software / plugin into Illustrator. I will be demoing some other software and will see how rendering 3D boxes is handled.
Everything 3D on my M3 Max so far is faster than my 16-core Mac Pro with W6800X Duo. That probably does not mean a lot today, but at the time of W6800 release, Mac Pro with Dual W6800X Duos was basically king for a day or two in Octane rendering performance.
I do believe there is a pattern to their hardware roadmap. They have enough nominal compute and register bandwidth to deliver a huge increase in matmul performance. My hunch is that they are planning symmetrical FP pipelines with packed matmul support - that would improve BF16 matmul thoughtput by a factor of 4x. The thing is, their architecture does not make much sense to me unless they plan to expand the pipeline capabilities. Concurrent FP32+FP16 execution is not that useful IMO. They are trading a lot of die area for minor efficiency and performance upgrades. But if they intend to make the pipelines more symmetric in the future the entire design becomes very smart. And the beauty of it - Apple doesn’t even need dedicated tensor cores. They already have enough pipelines in their GPU cores to compete with Nvidia’s tensor cores. What they need is the capability to invoke all these pipelines simultaneously for matrix multiplication.
Likely a software optimization issue. Apple GPUs are very good in rasterization workloads.
It definitely is a software issue. I am demoing some competitors software and it is as smooth as a hot knife through butter doing the same thing, but larger and higher resolution.
Nvidia might be in a bit of a bubble at the moment, and the reason I say that is because I believe the tech that's causing that bubble (AI) is in its own bubble right now. A bubble within a bubble if you will.
I believe generative AI like Chat-GPT or Gemini can be very useful--I even subscribe to the higher tier Chat-GPT plan. But it's far less useful than the tech bros and marketing people want you to believe. And guess who's going to get dinged for that the most soon? You guessed it--Apple.
I believe generative AI like Chat-GPT or Gemini can be very useful--I even subscribe to the higher tier Chat-GPT plan. But it's far less useful than the tech bros and marketing people want you to believe. And guess who's going to get dinged for that the most soon? You guessed it--Apple.