when a chatbot becomes a chatgod
Thanks to out of control industry hype, papers which question the capabilities of LLMs are provoking visceral responses. They shouldn't.
Did you hear the disturbance in the force? As if millions of AI bros suddenly cried out in fury and then took to social media to call Apple a bunch of losers? Yeah, that’s been a thing that happens on a regular basis now as machine learning experts who work for Apple kept publishing papers exploring the limits of large language models. The last in the series found that LLMs struggled to solve basic problems, which means that they can’t actually reason and problem solve despite viral claims by AI maximalists.
You can understand why they’re so upset. If our future AI overlords can’t play a Tower of Hanoi puzzle game, something that primitive neural networks from the 1970s could do without breaking a sweat, how are they going to, you know, overlord over us by the glorious Singularity of 2045? Cue the cries of “Siri sucks” and “Apple is overrated” as per the custom in reply.
Now, for full disclosure, I do use a Mac and an iPhone, but I’m absolutely not an Apple fanboy and agree that when it comes to innovation, they’ve been dropping the ball for a while. But at the same time, their whole thing seems to be that they will not try to be the first to do something, they will be the best at doing it. Hence they hired a team of researchers to figure out the limits of whatever new Siri they want to build without any reliance on ChatGPT.
If the pervasive rumors are true, Apple is playing the long game by figuring out how to run big AIs on an on-board chip, allowing you to take full advantage of an LLM without requiring you to leak sensitive data with opaque safeguards if those even exist protect you, which — let’s be honest, they probably don’t given what we’ve been seeing with too many models — as current chatbots demand you do. So, yes, it makes sense for a great deal of homework on how to miniaturize these AIs and what to expect of them.
out of many, many
With that out of the way, let’s return to the question of LLM reasoning and why it’s the hot topic on tech social media. Over the last few years, a lot of people fashioned their social media presence into being AI gurus who are always optimistic and always ready for the Next Big Thing, which is AI, of course. Even though AI is really just an umbrella term under the even bigger umbrella of machine learning, and LLMs are just one of an increasing variety of tools that can discover new antibiotics or predict the weather.
Again, it’s important to note that all these models above are not the same. They’re just different ways of deploying the same very useful statistical formulas with just a touch of integral calculus for error correction. While in social media, AI fast becoming akin to Tony Stark’s JARVIS, in reality, it’s becoming a design pattern in which you will just mix and match the models or chain them together to solve complex problems as tech bro keynotes keep promising digital immortality and a glorious future with the One Model To Rule Them All. Which is, coincidentally, the one they own.
As a result, the gap in public perception of the technology and its implementation has turned into a chasm. People whose identities are now wrapped up in the soon to be AI future where we can just type our problems into a text box and get instant solutions in easy to follow steps, like The Yogurt in Love, Death + Robots, listening to the endless streams of confident technobabble from AI startup CEOs and going further down this mental rabbit hole, just don’t understand how their chosen technology can fail such a simple task without there being some flaw in the study. Or in us.
This is why it seems so important to so many people invested in the narrative of AI as a single thing to defend LLMs from the implication that they’re bad at solving certain problems. Critics of the Apple paper already claim they found a workaround to get an LLM to “successfully” beat the Tower of Hanoi. Which involved just straight up telling it how to do that step by step and is being defended by a disturbing number of people who post that hey, maybe the LLMs are just proving we can’t solve problems either.
the one model to not rule them all
Meanwhile, the question in my mind is why does it matter if LLMs can solve the Tower of Hanoi and how? That’s not why they exist. They were created to allow computers to understand human input, which is often inconsistent and relies on implicit context and just so happen to have the useful side-effects of generating certain text and mapping relationships that exist within languages and common contexts. They’re not meant to be the Everything Machine and were never designed to do any such thing.
In fact, the whole notion that we keep pretending that they are, so much so that in the quest make their investors feel like they’re keeping up with the AI hype train, Google is destroying the web as we know it by deploying it everywhere and ignoring most of its errors and hallucinations, seems as absurd as insisting on assembling furniture with a hammer, insisting that screwdrivers, drills, Allen wrenches, and pliers are just obsolete tools of the luddite past and should be discarded.
There’s a bizarre obsession in Big Tech with the idea that one day, they’ll finally have a team of nerds crack the code and come up with the almost religiously prophesied AGI superintelligence that will usher in the Blessed Kurzwelian Singularity, and their CEOs and heads of R&D are either convinced that LLMs are it and we just need to double or quadruple down on them, or heavily invested in marketing their chatbots this way and now can’t backtrack with a far more realistic message.
And unfortunately, this pervasive, incessant, ever-escalating hype is turning questions of choosing the right tools for the right job when it comes to the future to AI into fiery social media arguments about LLMs in a way that’s eerily similar to creationists’ anger that someone doubts the infinite wisdom and abilities of their god. Which, by the way, a rather disturbing number of people are now routinely claiming to have discovered or created with chatbots…