the tech bro's guide to rediscovering everything
Tech influencers keep rediscovering decades old research in their quest to reinvent the world with only the power of hubris and a few lines of code.
Ilya Sutskever, the man widely credited with bringing ChatGPT to life, recently had a bit of a profound revelation. It's been a kind of tenet of faith that an AI can be far, far more intelligent than any human solely as a function of their raw processing power and a lack of weird distractions like emotions and urges. But it seems that in his so far seemingly fruitless quest to build this much rumored super-intelligence after raising $2 billion for the task, he discovered that emotions may actually be pivotal to making good decisions that make sense in a broader context.
While this sounds like a profound breakthrough, any psychologist could've told him the same thing. Without any emotions, facts and options are just that: facts and options. All your choices are more or less equal as long as they fit the context. In the example Ilya used, a man took hours to pick what socks to wear because all of a sudden, there was no signal to make a certain choice. All the socks fit, but not one pair stood out, not one pair felt more comfortable, not one pair was a favorite color or evoked a good memory. So, which one to pick? Did it matter?
Even something as simple and easy as "I really don't care, I'm in a rush, any pair will do" wasn't an impulse, so yeah, all of a sudden, basic decisions felt impossibly hard without the value signals that made some choices stand out more than the others, even if they weren't based on ironclad logic. And if they were, ironclad logic in a vacuum of just that decision, without the additional vast context guided by memory, intuition, or any long term plans and motivations, doesn't always result in good decisions.
He may have been referencing a case study involving a man named Elliot in the book summarizing the research, who had a brain tumor which affected his frontal lobe. The tumor was removed, but his emotions were shot. As a result, he was often, and rather befuddlingly, paralyzed by indecision or kept on making terrible ones, including a scam that left him broke and twice divorced. He aced every cognitive test but his ability to evaluate decisions beyond surface level meant that while a logical decision was easy for him, applying scrutiny to it was an now an impossible task.
To the denizens of Less Wrong, who think that Bayesian inference is the end-all-be-all of all rational thought, this probably seems absurd. After all, humans make a lot of weird, odd, and irrational decisions and work with deeply imperfect information. Surely, a cold machine armed with pure facts will be much better at this since it will be armed with all the data to make the right choice, right? Except, this approach presumes that perfectly informed agents can exist, something physics itself is iffy on at best.
Ask it to pick out your socks without giving it anything more to go on, it might pick a random pair that clashes with the rest of your clothes and find it immaterial that you're bothered by that because it doesn't feel self-conscious or understand that it's like to be self-conscious. If ti makes a meal prep plan for you, the fact that you will get bored or don't like one of the ingredients it picked won't slow it down since the cold, hard facts are: 1) this meal plan is optimal for you, and 2) you asked it to come up with an optimal meal plan by macros and calorie counts, and it did.
What you really end up with is at best, is a calculator that can give you some ideas or ask some very detailed questions to help you make decisions, but not one that could make those decisions better than you on a regular basis, or make those decisions with respect to others or openness to new ideas because yes, empathy and curiosity are all emotional at their core, and evaluating the outcome of a bad decision, or making peace with an imperfect one, but resolving to change the circumstances next time require one to have emotional awareness.
This is why Marc Andreessen, who demands an AI-first utopia from us, saying that he has no time or desire for introspection and doesn't intend to do any, is such a terrible sign of just how deeply ignorant about other humans tech bros are. If he doesn't feel regret or joy about a certain decision, like say his massive miscalculation about NFTs, he'll fail to learn where he went wrong and continue to make similar mistakes in the future, chasing after the Next Big Thing that no one actually wants.
For example, the AI that decided on a meal plan for you has to have some concept of empathy and want to do better, to decide on its own to ask you what foods you like and what foods you don't, how often you want variety, and if there is anything else it should know. Maybe even have a backup plan if you have an awful day and you really just don't feel like anything it made you prepare. If it's struggling to make a choice, it needs to be able to register that as discomfort, back off, and broaden its problem space to hopefully make a new and better one.
And again, none of this is new. Psychologists knew that emotions play a critical role in making good decisions for decades. Elliot's case study was published in 1994 and used work started before Sutskever was born. He is a victim of the same disease that plagues all of Silicon Valley, with Andreessen, Thiel, and Musk as its most prominent sufferers. He launches himself at a problem believing that as a comp sci expert, he can figure out a new way to tackle it, discovers how it actually works, then comes back down from the mount to teach us plebs as if he's made a revolutionary breakthrough.
People like this don't know how anything works and they don't care, sure that in their math and coding brilliance, they can easily revolutionize whatever the field is or solve any vexing problem befuddling experts. This is why DOGE was so chaotic and actively harmful, but ineffective. This is why Sutskever's AGI is not even close to happening. All he did was rediscover Psych 101 and the basics of embodiment theory. This is nothing more then professional Columbusing backed by an ocean of technobabble.
Back in the heyday of this newsletter's earlier iteration, the Less Wrongians and those who were committed to their mis-understanding of the Singularity bitterly and angrily argued that when I would talk about humans being more than just neurons sending electrical pulses to each other, I was engaging in some kind of spiritualism in which a biological system had some soul or essence impossible to replicate. You see, they had reduced our minds to integrated circuitry running on Bayesian logic and I was either trying to feel special by overcomplicating things.
Of course that's not at all what I was saying. Having taking psychology classes and with an interest in biology, I simply knew that we were just a lot messier and the arrogant over-simplification of that messy, convoluted system in constant flux will not be easy to replicate, and maybe even shouldn't be replicated, but is beneficial to fuse with if we want to transcend our biological limitations while retaining our agency and sapience. Making an app is easy. Altering the human species, on the other hand, takes a lot more than some C++ or Go ran on the correct chip or FPGA cluster.
When Sutskever quit OpenAI with great drama and fanfare, LLM boosters asked in a breathless whisper "what did Ilya see?" Back then, I would've said that he saw the limits of LLMs and knew he could just make them larger and larger while doing tactical node pruning, but the disruptive revolution was already accomplished, so he was happy to skedaddle. Today, I would say that he was caught up in his own hubris and really, truly, genuinely thought he could Less Wrong his way into an AI that's smarter than humanity and fulfill the Singularitarian dream.
