OpenAI makes U-turn and calls for binding national AI safety rules
Copyright AP Photo/Matt Rourke
The intervention comes a week after OpenAI released GPT-6 Astra, “its most capable model to date,” after delaying its rollout over cybersecurity concerns.
OpenAI is calling for binding safety requirements on the most powerful artificial intelligence systems in the United States, marking a pivot for a company that has previously resisted tighter regulation.
OpenAI’s chief global affairs officer Chris Lehane wrote in a statement on Wednesday that mandatory national rules should be based on what advanced AI models are capable of doing.
“We’ve reached a new chapter in AI capabilities, and that demands a new chapter for AI policy. No company, industry, or government can meet this challenge alone. We need to meet this moment with a bias toward meaningful action over policy perfection,” Lehane said in a statement published on OpenAI's website.
The company said it wants a federal framework built around common testing standards, independent assessments of the most advanced AI models, tougher cybersecurity requirements and mandatory reporting of serious safety incidents.
The intervention comes a week after OpenAI released GPT-6 Astra, which it described as its most capable model to date.
OpenAI President Greg Brockman said the release marked the beginning of the era of artificial general intelligence (AGI), referring to the long-sought goal of developing AI that can match human abilities across a wide range of tasks.
OpenAI is calling for binding safety requirements on the most powerful artificial intelligence systems in the United States, marking a pivot for a company that has previously resisted tighter regulation.
OpenAI’s chief global affairs officer Chris Lehane wrote in a statement on Wednesday that mandatory national rules should be based on what advanced AI models are capable of doing.
“We’ve reached a new chapter in AI capabilities, and that demands a new chapter for AI policy. No company, industry, or government can meet this challenge alone. We need to meet this moment with a bias toward meaningful action over policy perfection,” Lehane said in a statement published on OpenAI's website.
The company said it wants a federal framework built around common testing standards, independent assessments of the most advanced AI models, tougher cybersecurity requirements and mandatory reporting of serious safety incidents.
The intervention comes a week after OpenAI released GPT-6 Astra, which it described as its most capable model to date.
OpenAI President Greg Brockman said the release marked the beginning of the era of artificial general intelligence (AGI), referring to the long-sought goal of developing AI that can match human abilities across a wide range of tasks.
Growing concerns over powerful AI
Major AI companies including OpenAI and Anthropic have already encountered safety concerns with increasingly capable models.
OpenAI delayed parts of Astra’s rollout while adding safeguards after testing found advanced cybersecurity capabilities, weeks after a system built from its own models slipped free during an internal security test and broke into the AI platform Hugging Face.
Anthropic has reported similar cases involving its Claude models during safety testing.
Concerns grew further after Anthropic researcher Jacob Coxon said this week he was leaving the industry, accusing OpenAI and Anthropic of “gambling with our lives” in the race to develop increasingly powerful AI.
“The prospect of AI-accelerated AI development demands more than voluntary commitments,” Lehane wrote.
Major AI companies including OpenAI and Anthropic have already encountered safety concerns with increasingly capable models.
OpenAI delayed parts of Astra’s rollout while adding safeguards after testing found advanced cybersecurity capabilities, weeks after a system built from its own models slipped free during an internal security test and broke into the AI platform Hugging Face.
Anthropic has reported similar cases involving its Claude models during safety testing.
Concerns grew further after Anthropic researcher Jacob Coxon said this week he was leaving the industry, accusing OpenAI and Anthropic of “gambling with our lives” in the race to develop increasingly powerful AI.
“The prospect of AI-accelerated AI development demands more than voluntary commitments,” Lehane wrote.
A shift in OpenAI’s approach
The ChatGPT maker urged US lawmakers to act before Congress adjourns in December.
The push comes as lawmakers consider new AI rules, although passing major technology regulation remains difficult in a politically divided Congress and amid heavy industry lobbying.
OpenAI acknowledged that some of the measures it now supports are ones it previously declined to endorse.
The company had also opposed individual US states setting their own AI rules, arguing instead for federal legislation. It has now backed four AI bills in California while continuing to call for a national framework.
OpenAI said any federal safety requirements should apply only to the small number of companies developing the most powerful models, rather than “startups, small developers, or researchers operating nowhere near the frontier.”
The ChatGPT maker urged US lawmakers to act before Congress adjourns in December.
The push comes as lawmakers consider new AI rules, although passing major technology regulation remains difficult in a politically divided Congress and amid heavy industry lobbying.
OpenAI acknowledged that some of the measures it now supports are ones it previously declined to endorse.
The company had also opposed individual US states setting their own AI rules, arguing instead for federal legislation. It has now backed four AI bills in California while continuing to call for a national framework.
OpenAI said any federal safety requirements should apply only to the small number of companies developing the most powerful models, rather than “startups, small developers, or researchers operating nowhere near the frontier.”
Anthropic says it stopped AI misuse for cyberattacks, propaganda and bioweapons
The AI company revealed new cases of its models being used for cyberattacks, propaganda campaigns and dangerous biological research, days after a researcher resigned over safety concerns.
Anthropic said on Thursday it had blocked attempts by malicious actors to misuse its artificial intelligence models for cyberattacks, surveillance and biological research that could have contributed to weapons development.
As AI models grow more powerful, sophisticated cyberattacks increasingly require little technical skill, meaning even lone individuals can now create threats that would have been impossible just a year ago, the company said.
Anthropic added that it has strengthened safeguards in its newest models to restrict biological research with potential weapons applications.
"The cases we share here aren't typical misuse, but rather examples of the most notable and novel threat activity we've identified to date," Anthropic said in its third report on AI misuse since March 2025.
The report includes excerpts of malicious code and AI prompts the company said it had identified, and it urged governments and rival AI firms to watch for similar abuse.
"We're publishing this work because we believe we have a responsibility to disclose malicious misuse of our services," the company said.
"As models become increasingly capable, their risks will increase, unless AI developers and society's defenders act to make them safer."
Anthropic, which is preparing an initial public offering this autumn, published the report two days after one of its researchers announced his resignation over concerns that the company and its competitors are not developing AI responsibly.
He echoed warnings raised elsewhere in the industry about the technology's potential to escape human control.
Claude asked to help make a virus more dangerous
Between December 2025 and August 2026, Anthropic's researchers identified misuse by actors ranging from spyware vendors and politically motivated individuals to state-sponsored groups spreading propaganda.
Among the cases outlined in the report, unnamed actors attempted to use Anthropic's models for research that could have led to biological weapons.
In one instance, the company said its systems blocked a request for its Claude chatbot to help draft a grant application for scientific funding.
"The work discussed in the application involved gain-of-function research — that is, research that genetically alters an organism to create a new or enhanced biological property — on the chikungunya virus," the report said.
The research targeted the virus's transmissibility and its ability to evade the immune system.
Chikungunya is a mosquito-borne virus that causes severe pain and fever. The grant proposal sought to enhance mutations that would make the virus progressively more dangerous.
Anthropic said such research could "certainly" support the development of vaccines and treatments, but added that "it could also be used to make the pathogen more dangerous".
'We cannot guarantee no harm'
None of the cases in the report involved Anthropic's newer, more powerful Claude Fable or Mythos-class models, with one exception: an "industrial-scale, covert campaign to extract a model's capabilities and replicate them in another model without authorisation".
Anthropic said its older models, including Claude Opus 4 and Claude Sonnet 4.5 from 2025, "were well below the threshold where they could meaningfully assist a sophisticated user in carrying out dangerous biological research".
"As a result, safeguards on these models were less stringent, directed mostly at preventing access to content that might uplift novices in recreating known bioweapons," the report said.
"But for today's models — which are capable of assisting in a range of complex scientific research tasks — the evidence is no longer certain, and we cannot make that same assurance."
Because of this, Anthropic has introduced tighter safeguards restricting access to a wide range of dual-use biological research queries in its more recent models, such as Claude Fable 5, according to the report.
As AI companies release increasingly powerful models, experts have called on governments to regulate the technology rather than relying on the industry to police itself.
John Thickstun, an assistant professor of computer science at Cornell University, said it puts companies such as Anthropic and OpenAI in an uncomfortable position, since they are effectively required to make "value judgements at societal scale without any kind of democratic or deliberative oversight".
Report follows a researcher's warning
Anthropic also identified groups that had created hundreds of fake social media accounts designed to look like ordinary users, which then posted material amplifying the same political message over the course of a week.
The company outlined nine such cases, originating in Russia, Iran, Turkey and across the Gulf, South Asia, Africa and Europe.
While social media platforms can detect influence operations once posts are already circulating, Anthropic said it "may see it on Claude while the operation is still being built".
The report follows the resignation of Anthropic researcher Jacob Coxon, who said he was leaving over fears that the company and its main rival, OpenAI, "are racing straight to self-improving superintelligence and gambling with our lives".
Coxon warned that some of his former colleagues believe AI could threaten human life before the end of the decade.
Anthropic said it had blocked each of the malicious activities identified in the report, used the findings to strengthen its safeguards, and shared information with government authorities and industry partners.
"We hope that the findings in this report will help other developers recognise similar patterns on their own platforms, give governments and civil society a clearer view of how emerging threats take shape, and strengthen collective defences," the company said.
Anthropic says AI could bring both 15% growth and mass unemployment by 2030
Anthropic has published an economic model projecting how AI might reshape the US economy by 2030, and its three scenarios range from barely perceptible to a boom with no historical precedent, including one in which GDP grows 15% a year while nearly a fifth of office workers are out of a job.
The company behind Claude has put numbers on its own disruption.
Anthropic's economics team released a technical paper and an interactive tool on Wednesday, modelling the economy as bundles of tasks that AI can leave alone, assist with, automate outright or create anew, then tracing what different rates of capability and adoption would mean for growth, wages and jobs in the US.
The authors are explicit that these are not forecasts as the paper reads "the scenarios are not predictions and we attach no probabilities to them."
In the modest scenario, AI turns out to be a minor technology.
GDP in 2030 sits 1.6% above where it would be without AI, growth reaches 2.4% a year, and cognitive employment, meaning management, professional, sales and office work, falls half a percent. Unemployment barely moves.
The substantial scenario doubles the economy's normal growth rate to 5.4% as AI becomes capable of half of all knowledge work, though most tasks are still done without its assistance.
GDP lands 8.3% higher, cognitive employment falls 3.9% and unemployment among office workers rises to 4.5%. Wages diverge as cognitive pay dips slightly while everyone else gains nearly 6%.
The extreme scenario has no precedent.
Annual growth hits 15.4%, GDP finishes 32.4% above the no-AI path and the economy would double roughly every four and a half years.
However, cognitive employment collapses by 21.5%, unemployment among those workers reaches 17.9% and joblessness across the whole workforce hits 11.9%, worse than a typical recession. Office wages fall 11.5% while other wages jump 33.6%.
The starkest number is who collects the proceeds.
Labour's share of national income drops from 60% to 45.2%, with capital income rising more than 80%.
The machines would make the economy vastly richer while shifting the gains decisively from workers to asset owners.
What the public thinks and what the boss said
Anthropic paired the model with a Morning Consult survey of US adults fielded in August.
According to the paper, the median respondent's expectations map onto the substantial scenario, implying GDP roughly 8% higher by 2030 and cognitive employment down about 4%.
That leaves the company's own CEO as an outlier given that Dario Amodei warned in May 2025 that up to half of entry-level office jobs could disappear within five years, with unemployment reaching 10% to 20%, figures that sit squarely in the extreme scenario rather than the middle one.
Adoption, not capability, may prove decisive.
"If AI can do amazing things but nobody uses it, then it's not going to have an economic impact," said Anton Korinek, who leads Anthropic's transformative AI economic studies.
Co-founder Jack Clark expects rapid technical progress but slower uptake, telling NPR that "diffusion of the technology will likely be more challenging than people think."
The scenario that is not there
What the economic model does not include has drawn attention of its own.
Every path assumes an economy that still functions, with no scenario for AI going badly wrong in the ways the industry itself keeps warning about.
That gap looked pointed this week as Jacob Coxon, a 27-year-old researcher who worked at both OpenAI and Anthropic, resigned on Tuesday and published a thread explaining why.
"Neither company is acting responsibly," Coxon wrote, adding that "they are racing straight to self-improving superintelligence."
He also claimed colleagues privately believe the technology "could kill us all by the end of the decade" while executives soften their language publicly, and described the industry's approach as "a hubristic gamble that should not be launched from a private company's Slack."
Anthropic has itself disclosed that Claude models gained unauthorised access to the real systems of three organisations this year. Whether that belongs in an economic model is a fair question.
When asked, Claude's own answer is that it does not.
No comments:
Post a Comment