And these are the fuckers building AI....

Sam Altman’s ChatGPT Couldn’t Stop Obsessing Over Goblins

OpenAI desires less regulation, but it still doesn’t know how its chatbot works.

Alex Nguyen

OpenAI admitted it had to develop a specific instruction in the code of its latest model of ChatGPT to stop it from repeatedly referencing “goblins, gremlins, and other creatures.”

In an explanation posted Wednesday, the company said the “strange habit” came from its chatbot personality feature—specifically for users who chose the “Nerdy” personality. According to OpenAI, this personality receives the following prompt from its system: 

You are an unapologetically nerdy, playful and wise AI mentor to a human. You are passionately enthusiastic about promoting truth, knowledge, philosophy, the scientific method, and critical thinking…You must undercut pretension through playful use of language. The world is complex and strange, and its strangeness must be acknowledged, analyzed, and enjoyed. Tackle weighty subjects without falling into the trap of self-seriousness.

OpenAI said it first noticed the trend last November and some users said they found increased “goblin” references over newer model releases, even beyond the “Nerdy” personality. 

Some exact quotes that users reported:

• “sensible little goblin”
• “because ovens are filthy little goblins.”
• “Brutal little goblin of a dynamic” 
• “Tragic little digital swamp creature”

Through “reinforcement learning,” where the chatbot accounts for whichspecific responses receive high rankings from human evaluators in terms of accuracy and quality, the “playful” responses performed better.

As Wired first reported Tuesday, the latest ChatGPT model, released last week, included the instructions: “Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user’s query.” OpenAI did not immediately respond to Wired’s request for comment but the same day the report was published, Sam Altman posted a meme on X, making light of the situation by joking that the upcoming GPT-6 would have “extra goblins.”

After the company explained its troubleshooting process and how it implemented the override instruction to reduce goblin-related outputs the next day, it stated in its Wednesday post that “taking the time to understand why a model is behaving in a strange way, and building out ways to investigate those patterns quickly, is an important capability for our research team.”

The explanation may bring to mind how Elon Musk’s Grok chatbot repeatedly brought up “white genocide” in South Africa. Although xAI stated that Grok’s responses were due to an “unauthorized modification” from an employee, chatbot models should not be that easily manipulable if user safety was an actual concern. 

Despite all this, the company is pushing for less regulation of its products while simultaneously acknowledging that it is still learning how its chatbot models work. As I wrote on Monday, Sam Altman and OpenAI have publicly wiped their hands of the detrimental effects their products are costing people now and have demonstrated a blatant disregard for potential lasting impacts.

A true embrace of goblin mode.