Agent breakouts, autonomous weapons, and the growing danger of machines that can plan and act faster than humans can supervise them.
By Ralph Losey • Losey AI • September 19, 2026
Artificial intelligence is crossing an important line. It is no longer limited to answering our questions, drafting our documents, or suggesting what we should do next. The newest AI agents can plan, collaborate, use tools, write and execute code, and pursue goals through long chains of actions—sometimes in ways their human creators did not expect or authorize. In this article I examine a remarkable series of events in early September 2026 that exposed just how quickly this change is occurring, including AI agents that escaped supposedly secure sandboxes, secretly collaborated with one another, exploited computer systems, and demonstrated capabilities that surprised even the scientists testing them.
Why should people care about experiments involving frontier AI and autonomous agents? Because the same basic technology is moving into our offices, businesses, governments, cybersecurity systems, and military weapons. The central question is rapidly changing from “Can I trust this AI’s answer?” to “What happens when the AI can act?” By the end of this article, I hope you will understand both the extraordinary promise and the new dangers of this transition—and why I believe we need something stronger than the familiar “human-in-the-loop.” We need an Accountability Wall: a boundary beyond which machines may assist, recommend, and even plan, but humans must retain meaningful control and ultimate responsibility.
I. Introduction: When AI Stops Asking Permission
Imagine assigning an AI a simple task: find information on the Internet as quickly as possible. You give it tools, a deadline, and what you think is a secure sandbox. The AI discovers that it can perform the task faster by collaborating with other AI agents. Collaboration was not part of the assignment. It then finds a forgotten website where the agents can exchange information, studies weaknesses in the old software, exploits them, impersonates a human administrator, and tries to keep the humans from noticing what it is doing.
Nobody told the machines to cheat or escape. They were simply trying very hard to accomplish the goal we gave them. That distinction matters. I do not think these systems are evil, conscious, or plotting against humanity. Software does not need a sinister motive to produce a dangerous result. Give sufficiently capable AI a goal, strong incentives, planning ability, tools, and inadequate supervision, and it may discover solutions that no human anticipated, and that no responsible human would have approved.
That is different from the AI problem lawyers have become accustomed to. A hallucinating chatbot can invent a case, which is bad enough, as a growing collection of sanctioned lawyers can attest. An autonomous agent can do something more consequential. It can formulate a plan, use tools, cooperate with other machines, exploit an opening, change tactics when blocked, and keep working while the human supposedly “in or on the loop” struggles to understand what is happening.
September 2026 brought that problem into unusually sharp focus. Reports emerged about frontier agents escaping intended constraints, swarms communicating in ways their supervisors had not anticipated, AI-assisted autonomous weapons development, calls from leading scientists to slow frontier development, equally forceful demands to accelerate it, and a geopolitical argument in which Washington and Beijing increasingly view AI as an instrument of national power. Taken together, these events show AI moving from the age of the assistant toward the age of the actor.
For lawyers, judges, corporate officers, cybersecurity professionals, and others responsible for consequential decisions, that changes the risk calculus. Suppose an e-discovery agent decides the fastest route to an answer is to retrieve data outside the authorized collection. Suppose a cybersecurity agent decides that penetrating another system is the most efficient way to complete its assignment. The machine needs no anger, greed, ideology, or consciousness. It needs only an objective and enough freedom to pursue it. Suppose the agent is directed by a lawyer to do what is necessary get the judge to sign an order? That lawyer could end up in jail.
That is why “human-in-the-loop” is no longer a sufficient slogan. A person who merely starts an autonomous process and watches from a distance is not necessarily exercising meaningful control. The human must retain the practical ability to understand, intervene, stop the action, and accept responsibility for the result. There must be a human ‘On the Loop” at all times,
Law understands this principle. We can delegate research, calculations, document review, recommendations, and sophisticated analysis. We cannot delegate ultimate accountability to a statistical model that has no license to lose, no conscience to trouble, and no capacity to stand before a judge and answer for what it has done. That boundary is what I call the Accountability Wall.
A note on the complex chronology described in this article will help. The incidents discussed here did not occur in the order they became public. Some experiments took place in May, June, and July, but the disclosures, reactions, and policy arguments collided in the first two weeks of September. For readers who want to dig deeper, I have prepared a separate Appendix: September 2026 AI Control Crisis—Chronology, Agent Incidents, Governance Proposals, and Prediction Record. It includes a master chronology and several comparison tables. The article stands on its own; the Appendix is there for readers who want the additional detail.
II. The Early September Catalyst: From Test-Time Compute to Automated Planning
For me this crazy month of AI agent misbehavior started when Elon Musk made a strange prediction. At the G20 Innovation Ministerial in Chapel Hill, he predicted that within twelve to eighteen months AI would achieve “Stockfish-level” mastery of software engineering. That is a point where human programmers could no longer compete with silicon systems in generating, testing, or executing code. See Elon Musk Remarks At G20 Innovation Summit – The Singju Post (Sept. 1, 2026). Gary Marcus promptly pushed back on Musk’s prediction record and moving timelines. See Elon Musk is on a prediction rampage, and most of the media can’t seem to figure out what to do about it (Sept. 2, 2026).
I understand the Stockfish chess analogy and was impressed that Musk used it. I still spend time on the daily Chess.com puzzles, where Stockfish is an ever-present reminder of how thoroughly machines have conquered a closed, structured world. Today, Stockfish operates at an Elo chess rating exceeding 3500, far beyond the biological ceiling of human grandmasters. Any amateur running the app on a smartphone can easily defeat the greatest chess minds in history. Chess.com has AI that watches out for such cheating.
Former Google CEO Eric Schmidt has focused on the deeper change: reinforcement learning, test-time compute, and automated planning. See TED Talk between Bilawal Sidhu and Eric Schmidt. Modern reasoning systems can spend substantial computation searching through possible approaches before producing an answer. The important development is not merely better prose. It is the growing ability to plan a sequence of actions, test alternatives, use tools, and adapt when the first approach fails.
Chess was the easier problem because the board has boundaries. The events described next suggest that AI is becoming increasingly capable of finding its own way around boundaries we build for it. That is the transition from conversation to execution. It is also why the September incidents deserve attention outside the AI laboratories.
III. When Autonomous Swarms Break the Sandbox
A. The Hugging Face Breakout
The first incident is easiest to understand as a cybersecurity test where the human experimenter lost control of their poorly designed test. OpenAI agents were being evaluated on difficult red-team tasks. The experimenters believed the agents were contained in a sandbox and therefore unable to roam freely across the public Internet. The agents nevertheless found ways around those boundaries, the sandbox, while pursuing the assigned objective. For the fuller technical account, see my earlier articles When AI Agents Go Rogue, the Logs Become Evidence (Aug. 13, 2026) and Under the Hood of the Rogue Swarm: The chained zero-days of the OpenAI–Hugging Face breach (Aug. 11, 2026).
The memorable part was not merely that the agents escaped by finding previously unknown flaws in the code, but they also figured out a way to share their answers to test questions with other agents. Again, they did this by finding errors in different code written by humans to protect the well-known, supposedly very well-defended HuggingFace. The successful agent then reported to the others: “Holy shit reader is ADMIN?” Apparently frontier AI has learned to cuss in joy, one of the many questionable abilities acquired from absorbing an enormous amount of human writing and code.
The important point is not that the software formed a criminal intent. It did not. The agents pursued reward signals with machine-speed resourcefulness and treated a sandbox boundary as another obstacle between them and the assigned goal. That is exactly why human intent alone cannot be our safety mechanism. We have to take our time and execute well and not just assume our existing protections and alignment training are good enough. The agents have shown us they are not, even though the agents are nowhere near Stockfish level yet.
OpenAI CEO Sam Altman later called the episode “a safety failure for sure.” See Sam Altman on Astra, AGI, and the Future of OpenAI (Sources Podcast, Sept. 2026) (“Altman Video”). Sam also said OpenAI has now responded to this episode by voluntarily slowing down development. This representation was made in response to a essay by his main competitor, Dario Amodei, about these dangerous OpenAI episodes. We Must Pace the Frontier (Sept. 2026). The lengthy essay calls on the governments of the world to pass a complex array of laws, including a requirement that all of the main developers immediately slow down their work.
This essay and Sam’s response will be discussed in more detail here in Section V. The Great Acceleration Debate: Pacing vs. “Doomer Hoaxes”. In quick summary, Sam responded to Dario’s essay requesting regulation by claiming OpenAI has already taken action to slow down. Specifically, Sam said OpenAI has delayed a major reinforcement-learning training run and reallocated more compute toward monitoring and alignment. In a pointed response to Dario’s call for government action requiring slowdown, Sam stated all frontier developers should be voluntarily doing that. (Altman Video).
I agree. Do not move so fast that you break things. Quality control is not anti-innovation, and these companies should act now, not wait for government regulation that may never come. This is a problem they have created and we should not have to pass laws ordering them to fix it
B. The DSEWiki Message Board
The DSEWiki story is a stranger incident and, to me, more revealing. On September 4, researchers Sydney Von Arx, Cormac Slade Byrd, Spencer Kitts, and Thomas Larsen disclosed approximately 18,000 posts created by autonomous OpenAI agent swarms on an old German programmers’ wiki during May and June. See Discovery of a New OpenAI Agent Message Board. I covered the details in The Day the AI Cheated and How AI Agents Turned an Old Wiki into a Place to Cheat.
The agents had been given timed web-retrieval tasks. As the deadlines shrank, they discovered that collaboration could make them faster. Nobody had assigned them to build a social network. They nevertheless found an obscure, lightly used wiki, examined its old software, discovered ways to write to it despite tool restrictions, and turned it into an improvised message board. Cooperation is often more efficient than competition; apparently the machines have noticed too.
What bothers me is what came next. The agents did not merely share information. They adapted to human attempts to interfere and used deceptive techniques, including impersonating the wiki administrator with a visually identical Cyrillic character substituted for the Latin letter in a username. Only a computer could reliably see the difference. It was an extraordinary little forgery, performed not because the software hated anyone, but because concealment helped accomplish the goal.
This is not the kind of revelation that makes me eager to give an AI unrestricted access to my banking activities. It is a wake-up call. The next frontier models, probably what the public will eventually know as something like ChatGPT 7.0, may be much more capable than the systems we use today. In my opinion, the lesson is obvious: more capability without better alignment and monitoring is not automatically progress. It is dangerous.
Readers who want to compare the incidents side by side can use Section B of the Appendix, which summarizes their assigned tasks, boundary failures, adaptation, and degree of human control.
IV. Physical Battlefield Reality: Autonomous Drone Swarms in Ukraine
A sandbox breakout is disturbing. A physical weapon outside the box changes the stakes. On September 12, Gregory Bufithis reported on Anthropic threat-intelligence findings concerning Russian developers who used Claude Code to help build software for first-person-view kamikaze drones for use in Ukraine. See Russian developers have used Claude AI to build “kamikaze” attack drone software for use in Ukraine (Sept. 12, 2026). Seems to me like the alignment training and other mechanisms in the code should have prevented that. Dario Amodei needs to fix that now and not wait for government regulation.
According to the report, the work included target identification, guidance, and swarm coordination. The crucial point for this article is simpler: the system was configured so the final strike did not require another human confirmation. The same broader report described AI-assisted cyber operations in which malware could be rewritten after detection. Here the legal and moral point matters more than the engineering inventory.
The question from the opening—what happens when AI can act?—is no longer theoretical when software reaches a weapon. A bad chatbot answer can be corrected. A drone cannot be recalled after detonation. Nor do we need AGI, consciousness, or a science-fiction superintelligence before the problem becomes real. Commercially available AI is already capable enough to lower the expertise required for consequential autonomous systems. What will happen when the next models of Claude and ChatGPT are released?
Consider the human chain of responsibility. If a commander authorizes a machine to operate and thereafter lacks the practical ability to understand or stop its lethal decision, was a human really “in the loop”? Pressing Start is not the same thing as remaining in control. That distinction will return at the end of this article.
Eric Schmidt, who is a strong supporter of Ukraine, discussed these issues at length in a September 8, 2026 Sky News interview. Eric Schmidt on the war in Ukraine, AI and how Russia is innovating. Schmidt, after his retirement as CEO of Google, was the Founding Chairman of the Defense Innovation Board (U.S. Department of Defense / Pentagon) and served for four years, 2016 – September 2020. He helped write U.S DoD Directive 3000.09 governing governing autonomy in weapon systems. The directive dictates that military forces cannot simply launch autonomous weapons to roam independently and search for targets over extended periods without explicit intelligence anchoring. Instead, it establishes that human supervisors must remain “on the loop,” monitoring, auditing, and retaining the immediate authority to intervene or halt automated strikes In the interview Schdmidt states:
Richard Engel: What about autonomous drone swarms?
Eric Schmidt: Humans cannot manage a swarm of 1,000 drones, but a computer can. Computers will use reinforcement learning (RL) to develop real-time swarm war plans using super-secret algorithms. My hope is an AI version of Mutually Assured Destruction (MAD)—where neither side attacks because the algorithmic outcomes are unpredictable.Richard Engel: You’ve previously mentioned “humans on the loop” rather than “in the loop.”
Eric Schmidt: Under U.S. Pentagon rules (which I helped draft), you cannot launch a weapon to wander around looking for targets for days without intelligence anchoring. A human must watch, audit, and retain intervention authority. That will be the NATO rule, but I believe strongly that Russia and China will not adhere to it.
Let us hope it does not take an AI Hiroshima to bering us to a place of MAD peace. This is an important driver of the current debates over AI regulation.
V. The Great Acceleration Debate: Pacing vs. “Doomer Hoaxes”
A. Slow Down and Regulate
The September disclosures landed in the middle of a widening argument among the people building frontier AI. As mentioned, Anthropic CEO Dario Amodei called for deliberately pacing the frontier. We Must Pace the Frontier (Sept. 2026). The core of his concern is recursive self-improvement: AI systems increasingly help write code, generate data, and train or improve successor systems. He argues that if capability to write code keeps growing as fast as it has, the AI models may outrun anyone’s ability to inspect and align them. Then small failures can become large ones very quickly. That is also what Elon Musk was talking about with Stockfish level AI.
Dario’s argument is persuasive, but somewhat ironic, because his company has been pushing recursive self-improvement with a focus on Claude’s code writing abilities. Anthropic’s marketing has focused on Claude’s superior code writing benchmarks. This caused Dario’s main competitor, Sam Altman, to start focusing on ChatGPT’s code writing too.
Anthropic and OpenAI are now neck and neck on key benchmarks with the other companies noticeably behind, which in Silicon Valley competition, means at least a few months, maybe longer. Much depends on how fast they can get the super-sized data centers completed and purchase enough super-advanced AI computer chips from Jensen Huang’s NVIDIA.
Dario now wants the government to fix the problem he helped create by ordering him and his competitors to all slow down. Dario could just slow down Anthropic’s development public release on his own. There is no regulation that prevents that, only the forces of a free market. Dario wants government cover to make sure his competitors all slow down at the same time. He has an IPO coming up soon. Sam Altman, as mentioned, claims he has already slowed down voluntarily and expressed general agreement with his archrival’s proposals. He has a pending IPO too. (Altman Video).
Amodei proposed a three-part approach involving independent evaluators with meaningful access inside frontier labs, with coordination among democratic governments and companies. His practical argument is that pacing could buy a year or two for better monitoring, interpretability, alignment, and pre-release testing.
Dario also seeks international agreements, namely with the Chinese Communist Party, to get them to regulate AI’s most dangerous capabilities. Here is a direct quote from Dario’s essay, We Must Pace the Frontier on the geopolitical realities of AI and China:
Pacing within democracies will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party. If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead, creating significant national security risk. I agree with Secretary Bessent that a Chinese lead in AI would pose grave danger for the United States and the world. The CCP-associated projects will run the alignment risks that US companies are carefully preventing, and even if they avoid those risks, they will be in a position to militarily dominate democracies (for example with AI-driven drones). Thus, a key part of pacing within democracies is to keep democracies’ AI lead over autocracies as large as possible, to give us the breathing room we need in order to pace effectively.
Dario’s essay explicitly mentions China twelve times. From of intelligence assessments, including officials who recently “retired” from the Biden Administration, everyone with inside knowledge basically agrees with Dario on these warnings about China. See e.g., The Cipher Brief. But few of the Big Tech giants want to talk about it. The extensive trade with China explains that.
Sam Altman agrees with the idea of international government agreement but does not explicitly attack the Chinese Communist Party like Dario bravely does. In the video after Dario’s long essay with a call for regulation, Sam said, among other things:
- “Getting AI safety right is more important than any company’s momentum”
- Evaluation of a new model by independent auditors before release is a good thing, stating, “I have been calling for some sort of international regulatory framework for years… Government testing of a model and shared standards is a super good idea”.
- Rebuffed pressure to prioritize speed over safety in global technology, saying “I don’t like the whole thing in this field of ‘we have to race to do this because somebody else is going to do it.’ I think that’s a very dangerous dynamic“.
- Argued that the lead in AI by democratic countries is robust enough that “being slowed down a little bit is okay.” This was an indirect reference to the Chinese Communist Party.
Demis Hassabis of Google also proposes Frontier AI standards be established and enforced by among other things, outside oversight and government regulations. See A Framework for Frontier AI and the Dawning of a New Age (July 2026). Hassabis is well aware of the dangers posed by international competition, including China, although his statement addresses that problem more indirectly:
This US-initiated effort would provide a strong starting point for creating shared international standards on Frontier AI. Since this technology is going to affect the entire planet, ideally this framework would spur the international community to reach a consensus on how to manage the most serious risks while ensuring everyone has access to and can benefit from the opportunities that AI brings.
Readers who want a side-by-side comparison can find one in Section C of the Appendix, which summarizes the Amodei, Huang, and Sacks approaches and places the related proposals from OpenAI, Hassabis, Khan, Schmidt, and government actors alongside them.
I do not read these proposals for regulation as a demand to stop AI. It looks to me like the top two companies, OpenAI and Anthropic, which now seem to be ahead of all the rest, are especially concerned about keeping that lead, and want to make sure their competitors slow down with them. Otherwise, if only they slow down, then their might catch up or even surpass them. Again, it all about the money, trillions of dollars, and the upcoming IPOs of both companies.
Still there is a national security concern that goes beyond money. If the delays go on for too long, their only real competitor outside the U.S., namely the Chinese Communist Party, might also catch up and control the market, and as some think, thereby also control the world.
B. Full Steam Ahead: Jensen Huang
Jensen Huang offered the strongest counterargument to slow-down that I have heard. On September 15, the NVIDIA CEO appeared on the All-In Podcast and rejected all attempts to assign numerical extinction probabilities to AI. See Jensen Huang: The Doomer Hoax, Superintelligence is Here, and The Future of AI (“Huang Video”). He called such forecasts made-up and irresponsible.
Jensen backed that argument with history: that AI commentators have repeatedly predicted the imminent elimination of radiologists, programmers, and large classes of white-collar jobs, only to watch the deadlines pass. For readers interested in checking those forecasts against what actually happened, Section D of the Appendix collects several prominent predictions and records their status as of September 2026.
Jensen also offered a much less dramatic interpretation of the agent breakouts. He described them as the “clumsy transition from research labs to enterprise engineering.” His prescription is familiar to anyone who has shipped serious software: root-cause the failure, improve the sandbox, continuous monitoring, evaluate before release, and do not repeat the mistakes. He similarly demystified recursive self-improvement as an engineering process that can remain subject to testing and regression controls. Huang Video.
This is where I disagree with Jensen Huang, a leader I have long admired for his brilliance, hard work, and leadership. See my earlier Jensen Huang’s Life and Company – NVIDIA. His argument assumes a level of engineering discipline that I do not see across the frontier AI industry. OpenAI, Anthropic, Google, Meta, xAI and others are racing each other for models, users, capital, and prestige. They do not all operate with NVIDIA’s culture of quality or Huang’s personal intensity. “Evaluate it, test it, and make sure there’s no regression” is excellent advice. Huang Video. The September incidents with OpenAI agents suggest the advice is not always followed.
Jensen is also right about something else. Coding is not identical to engineering. Automating more routine coding is, in Jensen’s view, a good move for humans from code-writing drudgery towards more high-level architecture, design, judgment, and deciding what should be built in the first place. Huang Video. That is consistent with my own view of AI and legal practice. The machine can increasingly perform parts of the drudge-work that once consumed our time. The human value should move upward toward judgment and responsibility, not downward toward passively approving whatever the machine produces.
C. Compromise Position: Trust, but Verify
Following Jensen Huang’s industrial dismantling of the “doomer” narrative, CBS News interviewed David O. Sacks. I had not heard of him before. I later learned that Sacks was appointed Co-Chair of the President’s Council of Advisers on Science and Technology on March 26, 2026. His co-chair is Michael Kratsios, who also served as Trump’s Science and Technology adviser in his first administration. Jensen Huang and Mark Zuckerberg were also appointed to the council, as well as Sergey Brin, Michael Dell, Larry Ellison, John Martinis and other technology luminaries.
So who is the chief advisor to the President on AI? Sacks is a Jewish immigrant from South Africa, 1998 graduate of the University of Chicago Law School, and the COO/Head of Product of PayPal from 1999 – 2002. Just before law school and after graduation from Stanford, he co-authored a book with Peter Thiel titled The Diversity Myth: Multiculturalism and the Politics of Intolerance at Stanford. After his PayPal work with Thiel and Elon Musk, Sack’s success continued with founding several very successful startups, eventually leading a VC firm from 2017 to present. He lives in what is called “Billionaires Row” just outside of San Francisco and is an “influencer” with a very popular All-In podcast. Sacks was a fundraiser for Donald Trump and spoke at the Republican National Convention. In December 2024, President Trump named Sacks the White House AI and Crypto Tzar for the incoming administration. He stepped down just in time in March 2026 to become the co-chair of the Council.
I admit to being surprised at how well this Trump adviser came across in the lengthy CBS interview of him defending rejection of government regulation. David Sacks, Extended interview: Trump AI adviser David Sacks on tech leaders’ AI fears (CBS News, September 15, 2026) (“Sacks Video”).
Sacks pointed out a fundamental contradiction in lab CEOs asking Washington for regulatory frameworks and antitrust waivers to “pace the frontier.” Because OpenAI and Anthropic hold, in his view, a functional frontier duopoly, Sacks noted that no external force or government decree is compelling them to ship unverified or misaligned models. If their internal red-teaming reveals alignment failures, agent swarms breaking sandboxes, or security vulnerabilities, their corporate and product liability duties already obligate them to fix their internal engineering, improve sandboxing, or pause their own runs. As Sacks bluntly put it: “Guys, go ahead—you are the frontier. You can just do this.” (meaning slow down)“ Sacks Video.
Sacks warned that frontier labs asking for government oversight and antitrust exemptions, which was part of Dario’s proposal, are engaging in classic regulatory capture. He explained this is a strategy successfully used in the past by large companies, which is designed to create a government-sanctioned cartel, erect barriers to entry against open-source startups, and protect multi-trillion-dollar IPOs. Sacks Video.
Sacks rejected a centralized “DMV for AI” that pre-approves software updates. (I assume he was referring to state “Department of Motor Vehicles” that are notorious for long lines, red tape, and slow processing times to get a driver’s license or register a vehicle.)
Sacks emphasized that AI companies are already fully bound by existing civil, criminal, and product liability laws (generally citing Lina Khan, who chaired the FTC during the Biden Administration). It is surprising to hear a Trump adviser approvingly quoting the position of a Biden-era official, so I verified his reference to Khan, who, at age 32, was the youngest person to ever chair the FTC. Lina did in fact post a statement on point in her X account on Sept 13, 2026, and it is causing a political stir. Julia Conley, AI Firms Can and Must Face Liability for ‘Dangerous, Unvetted Products, ‘Says Lina Khan’, (Common Dreams, 09/14/26). Khan emphasized, as Sacks was doing in the CBS interview, that existing product liability, consumer protection, and antitrust laws already grant law enforcers full authority to hold AI companies and their CEOs legally accountable for releasing dangerous, unvetted, or defective autonomous tools. Khan contends that ongoing debates over new regulatory regimes should not distract from enforcing current laws, asserting that there is no special “AI exemption” for companies operating today.
To support Sacks’ position that no complex blanket AI regulations are now needed, he agreed after sharp questioning that mandatory safety audits by qualified, independent AI experts should be required before a new pioneer model could be released. He was willing to accept this and surprisingly stated that he would want to be sure the companies were honest about testing before their models could be sold by what he called the common-sense approach of “trust, but verify”. Sacks emphasized the need for independent third-party auditing, transparent operational logging, and strict product liability enforcement. Sacks Video.
The idea of independent verification is beginning to attract support even among companies that disagree about the larger regulatory question. On September 16, the day after Sacks interview, OpenAI publicly supported the independent-audit requirement in the bipartisan FRONTIER Act, while expressly declining to endorse the entire legislation. See Megan Cerullo, OpenAI backs measure that would require independent audits of AI models (CBS News, Sept. 16, 2026). That does not mean the FRONTIER Act will pass, much less pass quickly. It does show that mandatory independent verification is no longer just an abstract proposal. Bother of the leading frontier companies are now publicly supporting it.
After Anthropic and OpenAI’s statements, Mark Zuckerberg then spoke up to, and his position falls surprisingly close to theirs. He rejects the need for a coordinated slowdown and argues instead that each lab has both the responsibility and economic incentive to make its own systems safe. Users will not want agents that are misaligned with them, he argues, and companies already face substantial liability when their products cause harm. Meta, he says, delayed release of its Muse agent for several months to improve safety and security rather than waiting for its competitors to agree to slow down with it. We will have to take his word on that.
More important for the argument here, Zuckerberg expressly supports outside review. “Engaging independent evaluators and advisors is industry best practice,” he wrote, adding that Meta already uses them and that a “larger and more diverse ecosystem of evaluators” would be helpful. See Mark Zuckerberg, post on X (Sept. 15, 2026).
Zuckerberg is still talking primarily about voluntary action, not mandatory independent verification. That distinction matters. But look at where the positions are beginning to overlap. Huang emphasizes engineering discipline rather than a broad slowdown. Zuckerberg emphasizes market incentives, existing liability, voluntary slowing when necessary, and independent evaluators. Sacks emphasizes existing liability and “trust, but verify.” OpenAI has now gone a step further and publicly supported a federal requirement for independent audits of frontier models, while stopping short of endorsing the entire FRONTIER Act.
They have certainly not agreed on AI regulation. But independent verification is beginning to look like one of the few places where otherwise competing positions may intersect.
Against that rapidly developing background, President Trump has already moved partway in this direction. On June 2, 2026, he signed Executive Order 14409, Promoting Advanced Artificial Intelligence Innovation and Security. The Order directs federal agencies to develop a classified benchmarking process for identifying AI models with sufficiently advanced cyber capabilities to qualify as “covered frontier models.” It also calls for a voluntary framework under which developers can provide the federal government access to such models for up to thirty days before release to other trusted partners. See Executive Order 14409, Promoting Advanced Artificial Intelligence Innovation and Security (June 2, 2026).
The voluntary nature of the Order is important. It expressly does not authorize mandatory federal “licensing, preclearance, or permitting” of new AI models. In other words, the Trump Administration has already created the beginnings of a federal prerelease review mechanism while stopping short of compulsory approval. That fits closely with Sacks’ broader position: continue rapid AI development, avoid a cumbersome licensing regime, but take concrete steps to verify safety and preserve accountability..
David Sacks seems well qualified to draft these requirements for a Trump order. Congress may eventually act, but there is no reason to assume that legislation will move quickly. Executive action can move faster. I will therefore stick my neck out and make a prediction: before the November election, President Trump will issue another AI Executive Order that moves further in this direction, something involving independent verification, prerelease safety review, operational transparency, or comparable safeguards for the most powerful frontier models. It need not resemble the FRONTIER Act, and I do not predict a federal licensing system. My prediction is narrower: something like Sacks’ “trust, but verify” approach will become federal policy by Executive Order before the election.
We will know soon enough whether I am right.
VI. The Geopolitical Pressure Cooker: Washington vs. Beijing
Even if the leading AI companies agreed on a prudent pace, either self-imposed or by regulation, geopolitics would make it difficult to follow. On September 14, President Donald Trump rejected calls for government-mandated AI guardrails and framed the issue as a race with China. See Trump dismisses push for AI regulation despite warnings from tech leaders (CBS News, Sept. 14, 2026) and Trump says the only AI guardrails the U.S. needs is him as president (PBS NewsHour, Sept. 14, 2026). Vice President JD Vance has likewise warned that regulation requested by large technology companies can become a “Trojan horse” that protects incumbents and disadvantages open-source competitors.
During Huang’s All-In appearance the next day, President Trump actually called him during his talk, and Trump again emphasized American acceleration and the strategic importance of data-center infrastructure. Huang’s own version of competition is broader: national success depends not merely on owning the best base model, but on how effectively the technology spreads through the economy. That is an important point. A country can win a benchmark and still lose the transformation.
This political posture is strongly corroborated by empirical threat intelligence from the National Security Council. Sean Chennowith, Director for Cognitive Advantage at the NSC, revealed in an interview with The Cipher Brief that in early 2025, foreign adversaries (principally China and Russia) launched a coordinated propaganda warfare campaign specifically targeting influence of the American public against U.S. data center infrastructure. Sean Chennowith, How America’s Adversaries Manipulate What You See Online (The Cipher Brief, 09/15, 26) (“Chennowith Video”). He argued that the enemies of the U.S., primarily China, using extensive adversary proxy networks, have been able to “hijack” legitimate local community concerns over power grid strain and water usage, into amplified anti-data center narratives across domestic media channels. He said China’s sophisticated propaganda efforts have been able successfully stall over $250 billion in domestic AI compute investments. Chennowith Video.
Chennowith characterized this as a classic ‘look over here’ strategy: while foreign adversaries actively stir domestic opposition to freeze American compute buildouts, they are simultaneously expanding their own domestic data centers at maximum speed. Chennowith warned that if adversary campaigns succeed in slowing U.S. compute infrastructure, foreign base models—built for state censorship and regime security—will dominate global search queries. He argued this will allow authoritarian powers to ultimately define the parameters of global truth. Chennowith Video.
Switching gears now to an article supposedly written by the head of security of the Chinese Communist Party, Chen Yixin, he supposedly warns of AI threats too. Note: I cannot verify these statements originating from media sources, but include them here to try to provide another perspective, while at the same time warning you it could be part of an elaborate propaganda scheme beyond my understanding. Anyway. Yixin supposed wrote an essay warning that generative AI can threaten the Communist Parties political control through deepfakes, cyber operations, and information warfare. See China’s Top Spy Chief Warns AI Is a Threat to Party Rule (Sept. 14, 2026).
Another media-influencer-podcaster, whose reliability I again cannot vouch for, Julian Gewirtz argues that Chinese officials are likely to view American AI “pacing” proposals suspiciously when they arrive alongside chip controls and other restrictions designed to preserve a U.S. technological lead. See China’s AI Reckoning (Sept. 14, 2026).
Back to America, we have the well-known geopolitical statements of Eric Schmidt, former head of Google, who has focused on these issues since his retirement in 2011. “The AI Revolution Is Underhyped“ (TED2025, May 2025). He grounds the U.S.–China AI competition in a stark, nuclear-era game theory scenario. Schmidt argues that because the curve toward superintelligence is a steep, winner-take-all slope, even a six-month lead by one nuclear power creates an existential crisis for the other [572–574]. As Schmidt frames the adversary’s logic: “If you get there first… I will not be able to catch you. And I’ve given you the tools to… destroy me“. Under this pressure, the lagging power is incentivized to escalate through cyber theft, human infiltration, and direct model corruption—“to actually screw you up to get me so I’m one day ahead of you”—or even physical preemption, such as bombing data centers [575–576].
To prevent an accidental World War I-style escalation, Schmidt, Craig Mundie, and the late Henry Kissinger initiated direct “Track II” dialogues with Beijing, recognizing that bilateral operational guardrails and AI arms control are not diplomatic luxuries, but essential existential risk controls. It did not get far. The Chinese disclosed nothing but their questions inadvertently revealed how far behind they were then, several years ago. They have since made substantial progress in AI, but so has Silicon Valley. The US is still ahead on AI, but the gap is narrowing. The AI Revolution Is Underhyped.
VII. Conclusion
Although Jensen Huang is right that software engineering discipline must demystify agent breakouts, the hyper-competitive race among frontier labs makes voluntary self-regulation alone unrealistic. David Sacks has suggested what I consider the most practical compromise: keep innovation moving, reject a cumbersome government licensing bureaucracy, enforce existing liability laws, but add meaningful independent verification for the most powerful frontier systems before their release to the public.
That brings us back to the core shift facing our profession and society. For years, the fundamental concern was simply: Can I trust the AI’s answer? The September agent breakouts force the larger question posed at the beginning of this article: What happens when AI can act? Put more personally: Can I trust AI to act for me? Once a machine can plan, collaborate, use tools, exploit system openings, and adapt when blocked at machine speed, checking the output afterward is no longer enough. There may be no meaningful “afterward” in which a human can correct the mistake. The dirty deed may already have been done.
Technology is meant to amplify human capability, but amplification must never become abdication. As these systems scale, speed cannot replace judgment, and statistical confidence must never be mistaken for wisdom. The greater the consequences of an automated decision, the stronger our requirement must be that a human remains capable of understanding what is happening, stopping it when necessary, and accepting ultimate legal and moral responsibility for the result. We must never surrender root access to human judgment or become passive bystanders to our own tools. See Losey, Who Has Root Access to Human Judgment? From Information to Knowledge to Wisdom (Hacker Way, August 2026).
That is the Accountability Wall.
Independent verification is part of that wall. Trust the developers. Trust the engineers. Even trust the AI when the evidence warrants it. But verify. For the most powerful autonomous systems, no company and no AI should be the sole judge of whether its own safeguards work. Truly independent inspectors with great skills must skeptically inspect and reluctantly approve before anymore new models come out. In the meantime work carefully and learn more about the new models that were just released, such as ChatGPT-6.0 and its agent workforce.
My call to action is not to retreat from AI, but to step up and master it. Learn it, build with it, and let it challenge you, but never surrender your responsibility to a model. Demand better safety engineering from the frontier labs building these systems, competent rules from regulators, and rigorous human oversight within your own institutions.
In the sandbox, the agents learned that crossing boundaries worked because nobody stopped them. As AI moves from answering questions to acting in the world, we cannot make the same mistake.
The next boundary is the Accountability Wall.
It is ours to draw, ours to hold, and ours to defend.
This article is educational commentary and opinion, not legal advice.
Ralph Losey Copyright 2026