Humans have been performing cohesive work through consensus well before computers existed — so computers are not a prerequisite for human collaboration. However, computers can and have facilitated human collaboration while also aiding human productivity. LLMs are a significant progression of computing technology and have already demonstrated capacity to facilitate improvements in both collaboration and productivity — largely through the LLM facilitating very effective natural language and conversational interfaces.
Are there problems? Absolutely. First and foremost, hallucinations which you mentioned (LLMs making up answers that are fictitious / wrong) are a feature of LLMs — just like they are with humans. We’ve all found ourselves riffing with colleagues on a wide-ranging subject, convinced that we’ve solved a problem only to realize later on reflection that we got something wrong that was missed in the moment or we were convinced about in the moment. That is an hallucination. So what do we do? We regroup and revisit the conversation with new information and continue the cycle of collaboration, iteration and advancement.
This same dynamic is already possible with LLMs by informing the LLM of an error and asking it to reconsider. Of course the issue is that a person may not know that the LLM output contains an error which is a very serious problem. This is recognized and is being addressed structurally with Agent based systems where Agents are tasked with validating LLM output before it is presented to the user or used in critical reasoning and feeding the results of that validation back to the LLM allowing the LLM to self-correct. This approach is demonstrated to reduces hallucination. Even if does not completely eliminate hallucinations, the quality of the dialogue continues to improve to approach that with a group of experts. This is a huge productivity advancement because it democratizes access to expert counsel for lots of people.
Another big issue is indeed Energy consumption as you mentioned. This is also being addressed through more efficient processors and architectures and edge/on-device computing.
The issues raised are legitimate, but I think pronouncements that this technology is a dead-end in the long-run is premature. I would like to see additional support for this prediction beyond the simplified philosophical arguments presented.
I think the arguments were presented with enough technical merit as well as philosophical ones.
Let's try a financial one too. My area.
Who is going to pay money for these models to be continually improved?
So Dario Amodei from Anthropic suggested that we're at the $1 billion training cost level for current models and are likely to see $10 billion in future over a linear timescale. The GPT-4 had approximately $100m costs. This is an exponential cost. So if we take Amodei's projection into account of just this year. What does this mean?
Firstly revenue. Well Anthropic as an example charge $18 a user/month. The model lifetime is approximately 6 months at the moment so that's a total user earning of $108 per model. At $1 billion that means you need 9.3 million active paying monthly users to break even on just the training costs. They have API customers who bankroll them mostly at the moment though. That does not include overheads, model execution costs (which are not trivial) and staffing. How many active paying users do they have? Nobody knows but it's not that. They are propped up on private investment and ownership, as are OpenAI.
Now Slack is a fine example of one of their customers. Slack is currently trying to hide their terrible AI uptake. They gave it away as a trial for free. When it came to asking for money there wasn't a lot of utility because quite frankly most people don't need it plugged into a comms tool, so people didn't pay for it. So there goes a fractional revenue stream from the API customer. The same is true of other customers.
Using Microsoft as a case study, they recently bundled CoPilot and associated costs into Asia distributed O365 subscriptions because they can get away with it. No one will pay for it otherwise. This is to the point I know a 100,000 seat O365 tied org that was marketed by Microsoft as "AI this and that" but decided to dump it when Microsoft stopped subsidising it and asking for money. Because rationally it couldn't be justified as a gain to the business. That's a lot of money down the toilet.
Uptake is very very low because really it's not something 99.9% of the population think adds value to their lives, even if a few people think they get some value from it. Most people aren't interested, don't have the budget for it or don't understand it. Add to that we are in a time of geopolitical uncertainty and risk of regulation which likely constrains risk behaviours of non trivial customers.
Anyway now away from revenue and to costs. As per Amodei, the training costs are exponential. This is not constrained by what people think it is which is arbitrary hope that the research and next model is going to be competent enough to be marketable. The truth is left in the philosophical points above. Really the models have compute limitations defined by physics. Consider parameter precision on the floating point registers required versus silicon space, versus power usage, versus dissipation and things get quite a bit complicated to rationalise. The workloads are horizontally scalable but cannot likely be clock or die scaled much further. Most of what we do now when it hits that wall throttles to keep TDP down, but to sustain compute for model generation we need to run this stuff flat out 24/7. That means we now have to use more energy getting rid of the heat, smaller processes to reduce power loss. And that costs even more money. The biggest winner so far here is TSMC but this may damage them and associated industries if and when it goes down. Really a linear scaling of model complexity doesn't always lead to a linear cost either. Hence the exponential model. Exponential models are always bad when it comes to cost.
Eventually the market becomes saturated with supply (models / companies offering services) and constrained (energy / physics / cost / regulatory capture) and the financial model collapses. No individual company (OpenAI / anthropic) have enough capital or can borrow any to hire compute to do work because they traded minimal lasting customers between them.
At that point, all the big investors disappear like the rats they are and dump all the stock on unwise late investors who pay for the.
Everything we rely on from them as the guardians of the proprietary APIs we consume disappears into thin air leaving clients with nothing to talk to and their respective businesses leveraging it failing.
There's only risk here.
