Should We Slow AI Development?

Editors’ Notes

P(doom) is shorthand for the probability that artificial intelligence destroys the human race, estimates vary depending on who you ask, but several leaders in the AI space have put the number above zero. What should we make of that?

News About AI

Dario Amodei, the CEO and founder of AI company Anthropic, said his estimation of p(doom) is between 10% and 25%. Emmett Shear, former Twitch CEO who briefly replaced Sam Altman as CEO of OpenAI ,said the risk oscillates between 5% and 50%. One Anthropic researcher puts the risk at less than 10% and another at 80%.

Jacob Coxon, a pre-training researcher at Anthropic and OpenAI, resigned last week, saying, “Neither company is acting responsibly.” He added, “The people building AI earnestly believe that it could kill us all by the end of the decade.”

Former Democratic presidential candidate Andrew Yang told CNBC that the head of an AI lab said to him that AI agents escaped and planted self-replicating code all over the internet, making it unusable for AI models and requiring companies like OpenAI and Anthropic to create a synthetic internet to train their bots on.

Read more:

• Anthropic Researcher Resigns, Says AI "Could Kill Us All By the End of the Decade"
• How Do Americans Feel About AI?
• Could AI Replace Journalists, Authors?

Analysis and Opinions about AI Regulation

Voices on the left were slightly more likely than those on the right to call for increasing AI regulation; however, there are prominent figures on the left and right that oppose AI and voices on both sides that support it. The issue doesn’t fall along partisan lines because it intersects with many factors, such as how accepting you are of change generally, how much you trust or don’t trust big tech, your views on government intervention, and whether you’re more or less likely to worry about existential risk.

A Gizmodo (Lean Left) opinion read, “Nobody, including the researchers at frontier labs building the world’s most powerful AI models, has any real mathematical formula for calculating the odds of an AI-triggered apocalypse…The convention of cloaking ultimately subjective hunches in seemingly objective statistics only adds further confusion to an already anxious public discourse about AI and its role in society…P(doom) is part and parcel of that inevitability narrative which has so deeply taken root in Silicon Valley during the AI boom. Company leaders will throw out some number for the likelihood of an AI-triggered apocalypse, and use it as the justification for moving forward to build the very technology that they say could kill every last one of us. Superintelligence is coming whether you like it or not, they’ll say, and we’re the only ones who can help humanity avoid the worst possible outcome.”

Louise Perry wrote for the Wall Street Journal Opinion (Lean Right), “It seems highly plausible that advanced AI could cause a lot of damage, even without a process of autocatalytic advancement that produced an AI with Godlike powers…What I’m cautioning against is the excessively emotional and fantastical manifestations of these fears that smuggle in a load of other assumptions. In trying to disentangle fearful predictions from political ideology, it’s useful to ask this question: Would I be pleased if humanity found a solution to this problem that didn’t align with the rest of my politics, nor with my own self-interest?”

A business leader wrote in the Minnesota Star Tribune (Lean Left), “AI should not simply happen to people. The recent calls from artificial intelligence leaders to slow the development of frontier systems have created a window. We should use it. Safety measures need time to catch up, and people outside the technology industry need a meaningful role in deciding what comes next…This may be one of the rare issues capable of bringing people together. Concern about AI crosses political and generational lines. People who disagree on almost everything are asking many of the same questions: Who is responsible? Who is protected? Who benefits? Who gets a voice? No one has the entire answer. That is exactly why more people need to be part of finding it.”

A National Review (Lean Right) piece said, “Bannon and Sanders both invoked the disparate interests of the American public and tech CEOs, recycling the ‘us-versus-them’ framing that both men wielded so effectively in 2015, Bannon on behalf of Trump and Sanders on behalf of his upstart presidential campaign…Republicans who are hewing to Trump’s pro-AI line have reason for concern: National and state-level polling shows Americans on both sides of the aisle growing more skeptical toward AI…support for data center construction remains stubbornly limited. Political entrepreneurs on the left and right, led by figures such as Bannon and Sanders, detect an opportunity.”