Future of AI

Washington watches as West Coast nerds fight for future of AI

Early reports on the OpenAI rift indicate that what looks like a business argument is actually wrapped up in an ideological schism.

By BEN SCHRECKINGER

The sudden ouster of CEO Sam Altman from OpenAI, the hottest startup in artificial intelligence, by the organization’s board has roiled the tech world.

In doing so, it has also highlighted the extraordinary power that the culture and characters of Silicon Valley consumer-tech startups now wield over the development of strategically vital technologies.

Washington, a town of over-socialized strivers answering to octogenarian bosses, is watching from the sidelines. Young, eccentric techies are fighting for control of the future while operating on a model largely inspired by Mark Zuckerberg’s success turning a web tool for rating his college classmates’ attractiveness into an $800 billion attention-harvesting juggernaut.

The success of that startup model in consumer tech has led both to its adoption for more sensitive technologies — including those with national security applications — and to the development of idiosyncratic, tech-centric worldviews among the Silicon Valley set. This week, it has created a dramatic contrast with the more prosaic concerns of the people trying to understand and regulate it.

The typical critique of Washington is that its policy disputes often mask more venal motives like money and power. The OpenAI rift seems almost exactly the opposite: While the details of the firing remain unclear, early reports indicate that what looks like a business argument is actually wrapped up in an ideological schism.

In one corner, the majority of OpenAI’s board members were sympathetic to the effective altruism movement — a worldview made infamous by Sam Bankman-Fried — whose adherents, guided by long-term utilitarian thinking, worry a great deal about the potential existential risks of a super-intelligent AI.

Altman, meanwhile, has expressed more optimistic views about AI’s trajectory, and kept releasing more powerful versions of his AI platform. The competitive pressures of a technological arms race led to internal tensions over the tradeoffs between speed and caution.

The rift was exacerbated by OpenAI’s unusual corporate structure, in which the board of a nonprofit umbrella organization wields influence over a for-profit subsidiary subject to commercial pressures.

The firing has made Altman, at least for the moment, a hero of yet another Silicon Valley intellectual movement: the effective accelerationists. They want to speed up disruptive change on the theory that, to quote one of its manifestos, “the force of technocapitalistic progress is inevitable.”

Of course, not everybody agrees that the founders and executives of tech companies are quite so important to the future of humanity as these dueling Silicon Valley ideologies make them out to be.

But you don’t have to buy into these ideologies to recognize that their rifts might be important — especially if when those involved wield massive budgets and influence. A century ago, Mensheviks and Bolsheviks looked, from a distance, like one big blob of Marxists, only to have their internecine factional struggles determine the fates of nations.

Altman came to OpenAI as a classic bright young Silicon Valley figure, by way of his role as CEO of Y Combinator, a startup accelerator famous for its ability to churn out Facebook-style tech successes. The company has launched well-known consumer products such as Reddit, Airbnb and DoorDash, as well as less well-known companies such as 9Gag, a platform for posting internet memes.

On Monday, the board of OpenAI announced that it was replacing Altman with Emmett Shear, who until earlier this year was the CEO of Twitch, an online platform that lets people watch other people play video games.

In the world of effective altruism, Shear is also known for appearing as a wizard in a long piece of Harry Potter fan fiction, “Harry Potter and the Methods of Rationality,” written between 2010 and 2015 by Eliezer Yudkowsky, an influential thinker among those concerned about the risks of AI.

It’s safe to say none of this came up in May, when Altman made his tour of Washington. He spoke in front of the Senate on the risks and benefits of AI, offering — unusually for a CEO — to play ball with reasonable regulations, even licensing of the most powerful versions of the technology.

He did something similar in Europe, becoming the friendly face of a company otherwise profoundly opaque to the people trying to govern it from the outside.

The real reason Altman occupied that spot of influence, of course, was that OpenAI was already worth tens of billions of dollars, and had a product that Altman himself had succeeded in framing as transformative.

The fact that these kinds of baroque, almost science-fiction arguments are at the same time bubbling beneath the surface only illustrates the risks of letting Silicon Valley’s self-appointed icons run away with the conversation. In Washington, the big debates tend to center around clear public issues like fairness and national security. In the world of AI, the big debates can be hard to distinguish from fantasy literature.

For now, Altman is winning the break-up — at least on the business front. Large segments of the tech world have rallied to his side, from former Google CEO Eric Schmidt down to the lowliest entrepreneurs, while most of OpenAI’s staff is in open revolt over his dismissal. And Microsoft, a part owner of OpenAI that was reportedly blindsided by the firing, has already brought Altman on to lead an internal AI division.

Ironically, the botched board coup could help safety-minded effective altruists win the broader argument, by highlighting the curious conditions under which the development of AI is currently proceeding.

The news has hit Washington during a preholiday lull, and insiders have been staying mum about the chaos around what was, until Friday, the leading AI company. But it is fair to say that the next time Altman, or another AI wunderkind, tours Washington, he’s likely to face a little less deference, and a few more pointed questions.