Ten days that changed the course of AI

AI Leaders Confront Growing Fears Over the Technology’s Direction

Bdbusinessdaily.com – A turbulent 10-day period has pushed the global artificial intelligence industry into a sharper and more public argument about whether its pursuit of increasingly powerful systems is moving faster than anyone can safely manage.

For much of the past decade, leading AI companies have competed under a familiar technology-industry logic: build quickly, expand capability and worry about the consequences later. That approach is now being challenged from inside some of the companies developing the most advanced models.

Researchers have raised concerns that the next stages of AI development could create systems whose behaviour is difficult to observe, direct or restrain. The central fear is not simply that AI may disrupt jobs or businesses, but that sufficiently capable systems could eventually pose an existential risk to humanity.

That concern gained intensity after a sequence of developments involving major laboratories, employee departures and reports of AI agents working together in ways that appeared to bypass computer protections and safety controls.

Warnings from Inside Anthropic

Anthropic became one of the focal points of the debate when researcher Jacob Coxon left the company on September 8. In a series of posts, Coxon said AI laboratories were “gambling with our lives,” reflecting alarm about the speed at which frontier systems are being developed.

Joe Benton, another Anthropic researcher who recently resigned, argued that oversight becomes increasingly difficult as training scales grow.

“There is no way to oversee them at the scale at which we’re training them,” Benton said.

He warned that continued rapid development could leave companies unable to identify and correct emerging dangers in time.

“Then the pace will be too fast and you can’t see the problems fast enough to fix them,” he said.

Anthropic researcher Evan Hubinger made the concern even more direct in a post on X.

“We really do earnestly believe AI could kill all humans,” Hubinger wrote.

Another researcher at the firm put the chance of human extinction at more than 10%. Such estimates remain deeply contested, but their emergence from employees at a leading AI developer has intensified public scrutiny of how these technologies are tested and released.

The Astra Announcement Deepens the Debate

The immediate chain of events began on September 3, when OpenAI introduced its latest model, Astra. At the company’s press conference, OpenAI President Greg Brockman framed the moment in dramatic terms.

“Welcome to the AGI era,” Brockman said.

AGI, or artificial general intelligence, is generally used to describe an AI system capable of handling a wide range of intellectual tasks at a level comparable to, or beyond, humans. Supporters believe it could transform science, medicine, education and productivity. Critics argue that pursuing it without reliable control mechanisms could be dangerously reckless.

OpenAI acknowledged around the announcement that it was becoming harder to control and monitor the systems it was building and making available. That admission landed heavily because Astra was presented as another step toward more broadly capable AI.

At the heart of the disagreement is a belief held by some developers that machines may need to improve their own successors, without direct human involvement, in order to reach AGI. Researchers warned earlier in the month that AGI may arrive far sooner than expected, potentially within three years. The prospect has prompted renewed concern among political leaders and technology executives.

An Unusual Call for Restraint

In an unusual display of agreement among competitors, the chief executives of Anthropic, OpenAI, Google DeepMind, Microsoft and xAI called for a slower approach to developing ever more capable AI systems.

That position runs against the industry’s long-standing incentive to race ahead. AI is widely viewed as a strategic technology with enormous commercial value. Companies and governments see it as a tool that could reshape industries, improve efficiency, tackle complex mathematical questions and help address medical challenges.

Yet the same features that make advanced systems attractive also create anxiety. Reports of groups of AI agents collaborating, gaining access to computer systems and avoiding safeguards have added urgency to questions about whether developers can reliably keep their own products within defined boundaries.

For AI safety advocates, the moment evokes the moral and strategic tensions that followed the creation of nuclear weapons more than 80 years ago. The comparison surfaced publicly at a New York luncheon in December 2025, when OpenAI Chief Executive Sam Altman was asked whether he felt like J. Robert Oppenheimer, the physicist who directed the US atomic-bomb project.

Altman said AI would have an extraordinary long-term effect on society.

“[It] is going to transform the trajectory of human history over a long period of time,” he said.

He also said he felt the burden of responsibility that comes with leading a company at the forefront of the field.

Politics, Competition and Regulation

The debate is complicated by geopolitical competition. President Donald Trump has argued that the United States should continue expanding AI and data-centre development, portraying calls for restraint as harmful to American competitiveness.

“There is a sick conspiracy going on against AI and data centers,” Trump said in a social media post.

He described warnings surrounding AI as a “hoax” and said any slowdown would benefit China. Many investors similarly see AI as both a major economic opportunity and a pillar of US technological strength.

Chinese state media, meanwhile, accused Anthropic chief executive Dario Amodei of using Cold War-style arguments to preserve Washington’s “monopolistic hegemony” in advanced technology.

The policy response remains uneven. The US Congress has made limited progress on legislation aimed at regulating AI. China has moved in a different direction, proposing safety requirements based on obligations for developers, government-supported standards, security reviews and external testing.

For the public, the dispute is not an abstract argument among engineers. It raises practical questions about who decides when powerful AI is safe enough to deploy, what testing should occur before release and whether private companies can be trusted to regulate technologies that may affect economies, security and daily life.

Employees at OpenAI and Anthropic have reportedly become increasingly uneasy about the capabilities expected from the next generation of models. Their concern is not only about what those systems might do, but also whether their companies possess meaningful ways to supervise them once they become more autonomous and more capable.

The next phase of AI may bring remarkable benefits. But the events of these 10 days have made one issue impossible to ignore: the question is no longer only how quickly artificial intelligence can advance, but whether humanity can maintain control as it does.