As AI leaders warn of catastrophe, US and China shun slowdown calls

As industry leaders urge a global slowdown in artificial intelligence development for the sake of humanity, the two superpowers driving the technology’s rapid advancement have given no indication they intend to hit the brakes.

The United States and China, which are locked in a fierce strategic rivalry, have each rebutted calls to slow the pace of AI development, even as industry executives and researchers stress the need for international cooperation to avoid exposing humans to catastrophic – or even existential – risks.

As US President Donald Trump and Chinese President Xi Jinping prepare to meet on Wednesday in Washington, DC, for the third time this year, expectations for cooperation on AI safety are modest. Observers anticipated measures that, at best, would fall far short of the coordinated slowdown that many experts said is needed.

“The two countries host nearly all frontier labs and most of the models people download,” said Alvin Wang Graylin, a senior fellow at the Asia Society Policy Institute’s Center for China Analysis, describing agreed guardrails between the US and China as “close to a precondition” for safe AI.

“A safety regime covering one side’s models covers half the problem,” Wang Graylin said.

Hampering the prospects for US-China cooperation, analysts said, are mutual mistrust and divergent perceptions of the primary risks involved.

Trump has dismissed fears about AI as a “conspiracy” and suggested guardrails are not needed under his leadership, stressing the need for the US to stay ahead of China in the frontier technology.

Trump, who has cast AI dominance as crucial to the US maintaining its position as the top global power, on Tuesday again poured cold water on hopes for international cooperation, telling the United Nations General Assembly that Washington would oppose any “globalist scheme” to control AI.

China has taken a more hands-on approach to AI regulation – releasing the third iteration of its security framework this month – but has similarly rebuffed calls for a slowdown, dismissing fears of existential risks aired in Silicon Valley as “fearmongering”.

While Chinese officials have raised concerns about the dangers of AI, they have related it to more prosaic concerns, including cybersecurity and the authority of the ruling Communist Party.

In a rare essay this month, Chinese Minister of State Security Chen Yixin warned of “hostile forces” using AI to “fabricate political rumours” and “incite antagonistic sentiments”, threatening “political security, institutional security and ideological security”.

“China also increasingly recognises loss of control, misuse and cyber-risks but is much more sensitive to the possibility that ‘AI safety’ could become a justification for restricting Chinese access to chips, computers or markets,” said Sun Chenghao, a fellow at the Center for International Security and Strategy at Tsinghua University in Beijing.

“So the deepest obstacle is whether both sides can separate genuine safety cooperation from the wider technology competition,” Sun said.

“If every safety discussion is linked to export controls or slowing China’s development, Beijing will naturally question the purpose of cooperation,” Sun added.

So far, the only proposals publicly floated by either side have been a US-China dialogue and a notification mechanism for reporting national security threats.

While those measures would be positive first steps, the US and China would still need to reach an agreement on the AI capabilities or uses that should be off-limits to avert the greatest threats, experts said.

There is no consensus within the AI sector about how the technology should be regulated – or even whether government-led supervision is needed at all.

While AI leaders such as Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have called for a slowdown and greater regulation, some experts argued that such calls do not go far enough and development should be paused outright.

“International coordination is absolutely critical. We need more than safeguards,” said David Krueger, an assistant professor at the University of Montreal who specialises in AI safety.

“We need an immediate, indefinite international moratorium on frontier AI development.”

Other key figures, including Nvidia CEO Jensen Huang and Meta CEO Mark Zuckerberg, have played down the need for external oversight and emphasised AI’s potential to deliver transformative benefits for society.

Among those most alarmed at the rapid pace of AI development, recursive self-improvement, which describes when AI models learn to improve themselves, is often identified as a critical tipping point.

In an essay this month urging the AI sector to “pace the frontier”, Amodei said the US and China should discuss an agreement to “substantially limit” the rate of AI development while acknowledging that any deal is unlikely given the potential for a rule-breaking party to shift the global balance of power.

“The US and China must agree on AI capabilities red lines that will not be crossed. Most crucially, AI must never get powerful enough for recursive self-improvement,” said Otto Barten, director of the Netherlands-based Existential Risk Observatory.

“We do not know where the positive feedback loop of ever-smarter AIs creating ever-smarter AIs without human supervision will end and what these agents will finally be able to do,” Barten added.

Wang Graylin said Washington and Beijing should come to an agreement on what constitutes an “unacceptable capability” and a common standard to test frontier models, ideally by an independent international body that is funded by multiple countries.

But Wang Graylin acknowledged that distrust is clouding the prospects for an agreement and US officials are likely to see any deal as impossible to verify.

“Verification is hard for software you cannot see,” he said.

Sun said the US could ease the path to cooperation by “drawing a clearer line” between policies aimed at AI safety and those intended to maintain Washington’s technological advantage.

China could also contribute to greater trust by providing “greater transparency about its safety practices, model-evaluation methods and incident-reporting mechanisms,” Sun said.

Henry Gao, an expert on China and trade law at Singapore Management University, said a deeper issue is a fundamental mismatch in how Washington and Beijing view AI risks.

“Many in the US AI-safety debate focus on the possibility of highly capable AI escaping human control and posing catastrophic risks to humanity,” Gao said.

“In China, national security concerns appear more geopolitical: A major perceived danger is that the US could gain a decisive AI advantage and use it against China,” Gao said.

“That creates pressure to accelerate rather than slow development.”

Some AI experts argued that the focus of a US-China dialogue should be on mitigating existing risks rather than slowing development outright.

“AI is already extremely useful and has the potential to improve lives around the world,” said Rui Ma, a tech industry consultant and the founder of the Tech Buzz China news site, adding that pausing development would also impact the technology’s positive applications.

“I think thoughtful rules of engagement, constant monitoring, open channels of communication and the ability to adjust our approach as AI acquires different capabilities are much more important,” Ma said.

“The safeguards need to evolve alongside the technology.”

Calling the upcoming talks between Trump and Xi a “positive signal”, Jiang Tianjiao, associate professor at the Development Institute at Fudan University in Shanghai, expressed hope the sides could overcome their geopolitical rivalry to find common ground.

“China and the US are currently the leading nations in AI development,” Jiang said. “As such, we bear the responsibility of major powers to prevent AI from spiralling out of control, and we should set a global example by cooperating to manage these risks.”