They will probably optimize to run it on iPhone and iPad first. For higher ending training or intensive inference on Mac, probably need 32-64 GB or more Ram anyways.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
LLMs: Why Apple may finally be able to upgrade base Macs to 16/512
- Thread starter senttoschool
- Start date
- Sort by reaction score
I heartily agree with the stockfish POV with caveats:
- Just as tech giants missed the ball on applying attention to reinforcement learning, we don't know what/when/if there will be another breakthrough in LLM training that might make it more economical.
- The future is what you make of it, and I'd say, right now, Apple is well positioned to leverage work in LLMs. Plenty of devices, brand loyalty, and an architecture that arguable could be the best for scaling to work with LLMs locally.
Let's hope nVidia and Apple (and others) stay competitive in this space. For now, I'm pretty happy to be able to tap into ChatGPT4 and its API.
I downloaded diffusion bee and mochi diffusion to play around with. When i tried to import a model, this is the first time my 16gb M1 has completely bogged down from RAM pressure. I had to shut down pretty much all the other apps.
Which goes to show the risk of buying what you need now vs leaving some headroom. An application i didn't even think about two months ago when i bought my M1 mini is suddenly something I'm interested in. Who knows what's coming down the pipe in two three years? I know that if i were buying a PC and didn't have to pay an arm and a leg for RAM upgrade, i would be getting 32gb minimum without a second thought
Which goes to show the risk of buying what you need now vs leaving some headroom. An application i didn't even think about two months ago when i bought my M1 mini is suddenly something I'm interested in. Who knows what's coming down the pipe in two three years? I know that if i were buying a PC and didn't have to pay an arm and a leg for RAM upgrade, i would be getting 32gb minimum without a second thought
Exactly this.
I thought 16GB was enough and I would wait for Apple to move the base for the Pros to 32/1.
But with the advent of AI, I will configure my M3 Max with 64GB or more.
I’m an early adopter - not your average Mac user - but I believe that when pure AI applications are more ubiquitous, even the masses will upgrade to more RAM.
Sure. This is currently the case.
But as we have seen throughout computing history, you first need a room full of computers to run an application. A decade later, you can run the same application on your phone.
For LLMs, it will be way faster than a decade. People are able to get a few smaller models to run on Android and iPhones already.
Investment into AI inference chips will drastically increase from here on out. I expect the Neural Engine to take up 50-80% of the SoC space in a few years. We used to call the CPU the brain of a computer. It’ll be the neural engine soon.
I think only hobbyists and researchers will want to run LMMs locally. For most people, we only run them for a few seconds per hour. Even if you are a writer and need the LMM the style/spell/grammar check your work and you work all day, you only run the model for a few minutes. The usage is VERY "peaky", where you need a lot of computation for a few seconds and then for many minutes, you need zero.
With high peak demands and then nothing, the most economical thing to to is to share the large computer. This is why the cloud "works", one large server can Support hundreds of users and the average load on the server is high. Keeping the average load high is the only way to justify the cost.
Today if I wanted local performance as good as I can get on the cloud I'd need to spend about $10,000 for a fast Xeon-based computer with an Nvidia RTX6000 GPU. Yes, the hardware price will come down over the years but the size of the LMMs we run will grow. It will ALWAYS be expensive to run these large models locally so there will always be more economical to share the large servers.
What we might do is locally pre-process the data. Certainly, it makes sense to tokenize the data locally. Maybe there is more that cane done locally. ut the model itself needs a lot of "compute" power.
With high peak demands and then nothing, the most economical thing to to is to share the large computer. This is why the cloud "works", one large server can Support hundreds of users and the average load on the server is high. Keeping the average load high is the only way to justify the cost.
Today if I wanted local performance as good as I can get on the cloud I'd need to spend about $10,000 for a fast Xeon-based computer with an Nvidia RTX6000 GPU. Yes, the hardware price will come down over the years but the size of the LMMs we run will grow. It will ALWAYS be expensive to run these large models locally so there will always be more economical to share the large servers.
What we might do is locally pre-process the data. Certainly, it makes sense to tokenize the data locally. Maybe there is more that cane done locally. ut the model itself needs a lot of "compute" power.
If Apple starts shipping all its machine with at least 16/512 it will be because they found a way to ensure the same margins and profit on the 16/512 base machine as an 16/512 machine that was upgraded from 8/256 on their build-to-order system.
Whether that means increasing the base price or decreasing manufacturing costs somehow, or both, they aren't going to let that money go.
-kp
Whether that means increasing the base price or decreasing manufacturing costs somehow, or both, they aren't going to let that money go.
-kp
It’s less about the cost of the base unit than it is about the profits from spec increases or loss of those profits if they are unnecessary. The marginal cost of 16GB is only a little more than 8GB of RAM when compared to the price of the device. Where Apple gets a lot of it’s high margins in when people spec up from a base 8GB to 16GB for $200. That is almost all profit as the increased cost to Apple of the larger RAM chips is small.
Should Apple increase the base RAM spec to 16GB, a lot of people will not be paying for upgraded RAM. Increasing the base price by $200 would cause more people to skip the purchase all together. They would need to be careful in making that transition. It would help if there were a clear need for more than 16GB of RAM and Apple could convince people to pay for it. Local AI processing might be one way to encourage that.
Yep. And Nvidia need to bump up the VRAM from 24 GB in 3090/4090 to at least 48-64GB in next iteration of mainstream GPU in 1-2k price range.
You’re right, it’s ridiculous that a little bit more RAM and SSD is supposed to make a Dell worth as much as a MacBook. I’d pay money to not run Windows, but apparently a lot of people pay premium for the privilege to use junk. When PCs were a lot cheaper and outdated every six months anyway, it was unsurprising that people preferred them over nicely designed luxury Macs. But at the same price point? You’ve got to be kidding me!
Your grasp of English is lacking. Dell has 4x RAM and 4x SSD, or as you call it - "a little bit more RAM and SSD". Combined with superior OS (preferred by a little bit more people - an order of magnitude to be exact) Dell probably should cost at least 4x more.
Your understanding and comprehension is rudimentary. People buy mac as whole device, both software and hardware. Apple isn’t selling RaM or disk space. They price a complete device in different configurations. If you think windows is superior, RAM and disk space isn’t relevant on Mac running an inferior OS for you.
Most people/businesses buy whole PCs too. There is also way more software on Windows. Apple fans simply can't buy Macs part by part. Apple won't allow them do it (keeping the margins high). That's the computer experience reserved for PC enthusiasts which many of them cherish very much. Building a computer from thousands of available components brings them joy. It is also the only way to get the computer in the configuration one really needs.
This isn’t 2010, most of the clients I work with, large or small now provide Macs, it’s been at least 10 years since I was forced to use a windows laptop by a client. Most companies have moved on to Saas model, they don’t want to install or maintain software on computers. Sure there are certain niche apps, but the gap isn’t as big.
Like I said Apple sells the whole system, they don’t price RAM or disk space separately. Apple subsidizes low end model and chargers premium on upgrades. If you are an enthusiast, Apple is not even an option.
SAAS or not, my corporate laptop has tons of locally installed apps (maintained centrally by corporate IT but some of it is managed by me, VSCODE being one example, monthly updates and no Saas for it). Comparison between cloud/browser based apps and native apps rarely (if ever) ends up with the win for the former. As far as companies offering Macs... There have been some changes there but not much. Just check the changes (or lack thereof) in Mac market share.
Also, if Apple was betting on Saas why would they even bother developing their own processors? Thin clients don't need much power (although, admittedly, nowadays web browser may use dozens of gigabytes of memory, Saas or not).
Clearly your company isn’t even target market for Apple. My last Max mini was 2009, Max pro was 2012 model. When it was time to replace my Max Pro, I went with thread ripper/3090 Linux/Windows Workstation, later upgraded to 4090. I love upgradability in my workstations, Apple wasn’t even an option.
M1 Max 16 MBP is entirely different beast, not many options on windows with sustained performance with out throttling nor anything matching unified memory. Apple has to build separate logic board for each memory capacity, be it 8/16/32/64. Basically you are paying premium for high end laptops.
I hope you don't think of 1TB of disk space as an awful lot? You need much more for even the smallest movie library and much less to speed up the crucial system files. More than 512GB is really unimportant to the user experience. That's why I don't mind running the base configuration.
And yet Dell struggles to uphold a measly 2.39% profit margin. Too much to go bankrupt, not enough to invest.
Me again preferring good and free software over nagging adware.
Steve Jobs genuinely believed a computer should be an appliance which just works, rather than a machine to tinker with. This wasn't true in the beginning. But over the long run, he was right. There are diminishing returns of self-built PCs. Price excesses, quality issues, design failures, horrible customer service, way too many CPU variants to choose from, crazy expensive motherboards, idiotic naming schemes etc...
I am a computer enthusiast, that's why I abandoned the PC after Windows 7 and two decades of running Windows.
No it doesn't. They are frustrated. And they make Youtube videos to state their feelings.
I'm all for increasing base RAM and storage, but I don't think will Apple will upgrade them solely because of LLMs. The target customer for base Macs are also not tech enthusiasts that would go out of their way to run LLMs locally.
It’s for enthusiasts currently. Absolutely. But it’s moving so fast that it wouldn’t surprise me if local LLM/AI applications move to the mainstream by next year.
This is what people missed.
Apple subsidizes entry price points with expensive upgrades.
It’s expensive to have premium screens, metal enclosure, top of the line speakers. It’s cheap to put on more SSD and RAM. Since PC makers don’t compete on quality, they just throw on extra RAM and SSD to compete.
RAM and SSD are components which Apple can't make themselves and needs to buy at market prices. So its actually more expensive to build Macs with more memory and storage. But more importantly, every company needs to create different price points for their products, so that wealthier customers can pay more. Sometimes it's just the colour, which makes a black Magic Mouse a little more expensive. And sometimes products need to actually differ in features. So what else would you prefer to create low, mid and high tier? Slower processors? Lower resolution? Cheaper materials? RAM and SSD size are perfect, because if you stay within the limits, you don't even have a worse product. It's not a subsidy, it's great marketing.
Some printer makers build chips into their cheaper models, which make them print pages slower than the hardware could. That's some crazy ********!
For every YouTuber complaining about something, there's at least 50 non-YouTubers not complaining about that. That is one of the worst ways to measure the sentiments of a userbase as a whole. I built my gaming PC from scratch to make sure I had what I needed in the machine rather than what some bean counter at HP/Dell/iBuyPower/etc. thinks people need. Unlike those YouTubers seeking clicks through misleading video titles and thumbnails, I did enjoy the process despite a couple of self-induced errors (and one case that was not manufactured to ATX specs) that delayed the completion of the project.
So tell us, what did you pay for your custom PC build? If it’s more expensive than a Mac, I’ll declare the end of DIY PCs. The big vendors are going to die anyway.
I doubt 8GB of LPDDR5 RAM costs Apple more than $5 and 512GB SSD costs more than $3. [0]
My point is that premium speakers, screens, metal enclosure, touchpad, etc. These components should be more expensive than adding some inexpensive RAM and SSD. Every Macbook has these qualities. In order to subsidize these premium parts, Apple makes the 16/512 upgrade expensive.
There are different strategies. Some PC makers mostly make low-end cheap devices that you see in Walmart with no option to upgrade. They're just as is. Most aren't configuring laptops on PC makers' websites. They're just buying whole laptops in retail or Amazon. For these PC makers, adding $15 extra to get to 16/512 is a very easy way to compete in value.
[0]https://www.dramexchange.com/
Last edited: