One of Australia’s top intelligence agencies has warned that AI attacks could exploit the country’s old technology, as the government negotiates its guardrails on artificial intelligence’s rapid development.

The chief of Anthropic, developer of Claude, has warned AI bots could swarm the internet within a year and called to “slow the pace” of development with support from the head of OpenAI and Elon Musk.

Abigail Bradshaw, the director general of the Australian Signals Directorate, said her agency did not know how many AI agents were active on the internet.

“I think that prospect of getting some sort of definition or calculation of how these agents are out there is really remote,” Bradshaw said while speaking to the Sydney Dialogue summit in Canberra on Monday.

“And I would just say: a lot.”

She said Australian governments and businesses would have to spend “enormous” amounts to update their vulnerable outdated technology, taking offline systems that Australians had come to expect would be available all the time.

Anthropic’s chief executive, Dario Amodei, recently called for independent assessors to interrogate AI companies’ models and for US government permission for leading AI companies to coordinate without being punished for anti-competitive collusion. Bradshaw called for national security agencies to be involved in the independent interrogation.

The Greens demanded the government legislate mandatory safety testing and transparency obligations for the most developed AI systems on Monday but their motion for debate was rejected by Labor senators.

Labor’s Andrew Charlton, the assistant technology minister, told the Sydney Dialogue he took the companies’ warnings seriously but did not want them or the US to be responsible for setting the guardrails around AI.

“I don’t think we should care very much about what individual chief executives say or what individual executives claim they want to happen,” Charlton said.

Charlton said the Albanese government would set its forthcoming guidelines for AI companies to cover the range of “extremely dangerous” and more benign possibilities being forecast.

The laws are expected early next year. Anthropic’s chief lawyer, Jeff Bleich, told Monday’s summit the company had been “heartened” by the strong support for its calls and said Australia would need to play a role in setting the rules.

“I don’t think it makes any sense that something this important would be decided either by company executives or by leaders in only two countries,” Bleich said.

“It’s just important for the model developers not just for social licence, which is critical, but also some legal certainty.

“It won’t be easy to get these things done but it actually makes it better to come to Australia.”

But Bleich said Anthropic believed rules were needed within months as the technology developed at an exponential pace.

One of the factors affecting companies’ interest in Australia has been access to copyright-protected material produced by local creatives so it can train its models. The industry minister, Tim Ayres, told the summit the government wanted local training but remained unwilling to reduce copyright protections.

OpenAI’s vice-president for global policy, Ann O’Leary, said her company was in “regular” and “deep” conversation with the government about the copyright rules but was “very eager to have a bigger and broader relationship”.

“I don’t have to tell any of you how important Australia is from a geopolitical standpoint,” O’Leary told the summit. “We very much recognise that having a diversification of where we train is going to be important as you move forward.”

She said OpenAI wanted to ensure creators could keep creating.

Andrew Hastie, the Coalition’s industry spokesperson, warned the summit Australia would be vulnerable in future conflicts if it could not get advanced companies to locate themselves in Australia, calling for a formal deal with the US.

The government is already using Claude’s advanced Mythos model. Paul Robards, a data expert at Australia’s department of defence, downplayed the likelihood of Australia developing frontier technology.

“Without a sovereign AI frontier model here in Australia, we are going to be a supplicant state, not a sovereign state,” Hastie said.

“We need to work on a way to get a hyperscaler down under … I think working with the United States, through Aukus if we want to use that frame, is the pathway.”