Shock Resignation in the AI World: 'It Could Bring About Humanity's End'

A striking development has reignited security debates in the world of artificial intelligence. Former AI researcher Jacob Coxon, who had conducted research at OpenAI and Anthropic, announced his resignation, warning that rapidly evolving AI systems pose serious risks to humanity.

On September 8, Coxon stated on the social media platform X that technology companies are advancing toward self‑improving "super‑intelligence" systems without acting responsibly. His most striking claim was that some people working in the industry truly believe that advanced AI could threaten humanity’s existence.

"Do Not Underestimate the Power of This Technology"

Coxon, who highlighted the rapid pace of AI development, argued that future systems could surpass human capacity in many areas. He noted that these systems might have the potential to carry out cyber attacks, achieve rapid progress in scientific and technological fields, and acquire resources and power in the real world.

Coxon claimed that some AI developers and managers are seriously concerned about the systems that may emerge in the coming years.

According to the researcher, while more measured language is used in public statements, much more serious concerns are expressed in private industry discussions.

Striking Claim About OpenAI and Anthropic

Coxon also critiqued the approaches he had taken at OpenAI and Anthropic in the superintelligence race.

Coxon, who has highlighted that some employees at OpenAI have not sufficiently internalized the risks of artificial intelligence on the “scale of civilization,” argued that at Anthropic the risks are better understood, but the company continues to compete on developing safe systems before other actors.

According to the report, OpenAI and Anthropic did not respond to requests for comment on Coxon’s statements by the time of publication.

Security Debate Moves to Washington

Concerns about artificial intelligence control are not confined to the tech world; Senator Bernie Sanders and Representative Greg Casar of the U.S. House of Representatives announced a proposal to temporarily halt the development of advanced AI and to ban artificial superintelligence until comprehensive safety regulations can be created by federal regulators.

One of the central topics of the debate is the possibility that future artificial‑intelligence systems could enhance their own capabilities without human intervention.

OpenAI Chief Scientist Calls for “Extreme Caution”

OpenAI Chief Scientist Jakub Pachocki also highlighted the dangers that advanced AI could pose in a post published on September 6.

Pachocki noted that within just three years, language models capable of reasoning have become an increasingly large part of the economy; they can use computers, collaborate with humans and other systems, and conduct research projects.

Pachocki, stressing that advances in cybersecurity are creating new risks, said that “extreme caution” is required when developing AI. In contrast, he noted that advanced systems could be used in defense against threats that other AIs might pose, recording that there are reasons to support continued research.

Similar Warnings from Within Anthropic

Coxon's statements also received support from some researchers at Anthropic.

Anthropic researcher Evan Hubinger said he believes the risk posed by current AI models is low, but he is concerned about future super‑intelligence systems that could self‑improve. Hubinger said he rates the chance of a result that would threaten humanity's existence within the next decade at less than 10%.

Samuel Marks, who conducts security research at Anthropic, also expressed the view that AI could lead to very serious consequences even within the next few years. Marks argued that the commercial incentives and the idea of getting ahead of “less responsible” competitors drive companies to continue their work despite the risks.

The Critical Question in the AI Race

With Coxon's resignation, the long‑debated question has returned to the forefront of the tech world: Is there a way to safely control AI before it becomes more capable than humans?

On one side are AI systems that promise major advances in science, health, economics, and technology, while on the other are researchers worried that the same technology could become uncontrollable.

Coxon's and other security researchers' statements, however, pull the debate out of a distant future scenario and focus on the consequences of decisions that will be made in the coming years.