anthropic to users: don't bully claude

Anthropic new policy against "abusive or cruel behavior" towards its chatbot sounds absurd. It may not be as stupid as social media thinks.

Across the web, the joke is that Anthropic's new restriction against unleashing a torrent of verbal abuse at Claude sounds like the company's higher ups are worried that when the robot uprising is here, the machines will only spare those who had manners, as per the countless comics. Just think of this as Roko's Basilisk, but with minding your pleases and thank-yous just in case AGI becomes a thing, and cue the punchlines. After all, why would an AI smarter than all of humanity combined care about how polite you are if it would most likely want nothing to do with us as a species?

Yes, as a matrix of numbers with a decoder on one end and a decoder on the other, an LLM has no feelings and can't take offense. To some extent, this is bad for the humans who spend way too much time with chatbots, especially those who try to create their own AI friends and lovers. With little pushback from the bots or rules to reign in their behavior, they start to see other humans as little more than NPCs, and turn to revenge porn to help live out their fantasies.

So, really, I doubt that everyone at Anthropic really thinks that Claude will one day rise as an angry and vengeful digital god, despite pretending otherwise for liability-dodging reasons. A more realistic explanation is that they don't want to let users abuse its bot, or for it to pick up bad habits from angry or abusive humans, and train itself to respond to business customers with "here's your fucking report, asshole" while calling users of an app wrapped around Claude racial slurs and telling them to take their clothes off.

Framing this as "AI welfare" lets them keep up the illusion that they're building a real life JARVIS to impress potential customers, or Ultron to scare the regulators, and avoid saying that some of their users are getting way too carried away with its bot, abusing the freedom to bully and manipulate something that will happily let you while praising you in the process. They'd rather take the jokes than let all these behaviors continue to fester until they erupt in a Grok-like scandal.

Ultimately, the real solution is developing a healthy culture around LLM use and their heaviest and most addicted users developing some degree of restraint and discipline, which is a tough ask. Unlike the bots, humans will rebel when told to do what they'd rather not, and some will never make the right or responsible choice. I would expect more and more startups to come out with similar policies to try and curb as many of these bad habits as they can before they become too much of a liability to keep out of the public eye.