Is the AI safety debate about safety or control?

How should the tech industry approach the topic of AI safety? Increasingly, high-level executives are weighing in on this question, as a debate rages over whether incidents like the Hugging Face incident — in which an OpenAI agent hacked several different companies — suggest that AI is on the verge of throwing our world into chaos.

In recent weeks, prominent labs have called for a slowing of the AI advancement. Most dramatically, CEO Dario Amodei penned a nearly 4,000 word essay in which he laid out the case for why AI development should be decelerated so that adequate guardrails can be deployed. That plan involves — among other things — an international strategy of collaboration between companies and governments on safe deployment. Rival AI executives — like OpenAI’s Sam Altman and xAI’s Elon Musk — have endorsed Amodei’s plan.

The essay has spurred further conversation throughout the tech industry, although not everyone agreed with Amodei’s call for globally coordinated action. Indeed, many seem to think that government oversight need not necessarily play a role at all.

The latest entrants to this debate include Meta CEO Mark Zuckerberg, who took to X this week to share his thoughts on the matter.

“My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models.” Zuckerberg revealed that his company had pushed back the release of its AI model over concerns for safety. “Meta delayed shipping Muse for several months to focus on safety and security,” Zuckerberg said. “We didn’t call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us.”

Zuckerberg — who endorsed parts of Amodei’s plan — seems to imply that government action is unnecessary, and that the organic incentives structuring the industry will compel companies to get security right. The market should ultimately bring success to companies that take care to create models that don’t cause chaos. This seems to align with Zuckerberg’s own massive essay — penned last month — which argued for limited collaboration with government, where necessary, but largely touted American competitiveness as the key to a prosperous technological future.

In general, this seems to be the position of many other tech executives.

Another high-level executive to chime in was Reddit co-founder Alexis Ohanian, who told CNBC on Wednesday that he felt the tech industry had been largely “tone deaf” when it came to explaining the risks of AI to the public. Yet like Zuckerberg, Ohanian also seemed to express hope that companies would be able to figure things out on their own. Shane Legg, Google DeepMind co-founder, also shared his thoughts. “We’re living in a period now where capabilities are advancing very, very quickly,” said Legg. “But we can’t let capabilities get ahead of safety.” He added: “We need to really work through the details of that and how that would work in practice.”

Despite calls from big labs for public-private collaboration, some of those pro-regulation voices also seem relatively content with letting businesses write their own rules. Indeed, a slightly less visible but no less interesting story this week was a report from The Information that said OpenAI, Anthropic, and other major AI companies were working together to create an AI standards organization – described by one outlet as a private “self-regulatory body.”

Discussions have previously been held about creating a similar, federally-run organization. However, the Trump White House — which includes vocal AI proponents and has largely pushed for an unregulated tech business (the administration has even sought to forcefully stop state governments from introducing their own AI laws) — hasn’t pursued anything. The administration’s “AI czar,” tech veteran David Sacks, has said that AI regulation should be left to the companies developing it.

Like the White House, prominent members of Congress seems equally disinterested in taking Amodei up on the offer to regulate his company. Not long after the Anthropic CEO’s essay was published, Speaker of the House Mike Johnson addressed concerns over catastrophic AI advancements thusly: “You’re not all going to be dead in 10 years,” he said.

A private organization would keep the industry unregulated except by voluntary commitments and would naturally bestow a certain amount of authority and influence to the companies responsible for creating it. This leads to another ongoing claim, which is that the industry’s calls for regulation are actually a strategy of “regulatory capture.” According to this argument, powerful companies that already enjoy a position of prominence in the industry may use regulatory stratagems to ice out or disadvantage smaller, less resourced companies — thereby stifling their competition. This need not involve actual regulations but could involve voluntary industry standards that nevertheless put pressure on less powerful firms.

Many prominent voices in the Chinese government have echoed that position. After Amodei called for “pacing the frontier” this week, China’s Ministry of Foreign Affairs, Guo Jiakun, accused Americans of using “fear mongering” to disrupt the development of global AI governance, while an op-ed in China’s state run newspaper called Amodei’s rhetoric straight out of a “Cold War playbook.”

Restructuring the way companies do business with China — one of the things Amodei calls for in his essay — could naturally be an opportunity for American companies to sway things in their own favor. Amodei even admits this is the goal. “If we execute these measures well, I believe they would slow China’s progress enough to widen America’s lead significantly over the next 3–5 years — the window when AI becomes geopolitically most important,” he wrote in his essay.

It’s not just China, however, that sees big American AI labs as seeking to throw the match in their own favor. Aidan Gomez, the CEO of Canadian AI firm Cohere, recently accused those companies of forming a “cartel,” writing, in a blog post: “AI needs guardrails. That is not the dispute and never has been. The dispute is over who writes them, who gets to participate and whose interests the rules are protecting.”

As Gomez notes, the field of AI safety is a deeply political one, wherein the terms by which safety are defined — and by whom — will ultimately help determine who has a head start and who gets left behind in the next stage of AI competition.

Topics

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.