P
Back to articles
News6 min read

Anthropic's warning over AI self-improvement has a hidden message — accelerating development requires more compute before companies ever risk losing control of frontier AI models

Anthropic's report reveals Claude now writes 80% of its own code, but the hidden message is that compute capacity — not intelligence — is the real bottleneck on runaway AI development.

Source: Tom's Hardware
Anthropic's warning over AI self-improvement has a hidden message — accelerating development requires more compute before companies ever risk losing control of frontier AI models

Anthropic's Self-Improving AI Warning Comes With a Compute-Sized Catch

Anthropic, the AI company behind Claude, published a striking report on June 4 titled "When AI Builds Itself," revealing that Claude now writes more than 80% of the code merged into its own production codebase. The finding, presented alongside warnings about potential loss of human control over frontier AI systems, has sparked intense debate across the industry — but a closer reading reveals a more nuanced story about compute constraints, marketing timing, and the practical limits of runaway AI.

AI circuit board with brain illustration
The warning from Anthropic about AI self-improvement has an underlying message about the critical role of compute capacity. Photo by Steve A Johnson on Unsplash

Claude's Rapid Acceleration, By the Numbers

The numbers in Anthropic's report are striking. Before Claude Code reached research preview in February 2025, AI-generated code made up only low single-digit percentages of merged changes. By Q2 2026, the typical Anthropic engineer is merging eight times as much code per day as in 2024. On the hardest, least-specified coding tasks, Claude succeeded 76% of the time in May 2026 — a 50-percentage-point rise in just six months.

Perhaps the most dramatic stat comes from an internal benchmark: each new Claude model is asked to make training code run faster. Results climbed from roughly triple the original speed with Claude Opus 4 in May 2025 to about 52 times faster with the unreleased Mythos Preview model by April 2026. For comparison, a skilled human researcher typically achieves a fourfold gain in four to eight hours on the same task.

Anthropic was quick to add caveats. Lines of code are a poor proxy for meaningful output, the company acknowledged, and the eight-times figure almost certainly overstates the real productivity gain. The research-judgment study in the report, where models beat human choices 64% of the time, drew on only 129 moments deliberately selected because the human's choice had room for improvement — not a like-for-like comparison.

The Hidden Variable: Compute as the Real Governor

Data center servers for AI compute
Data center capacity, not AI intelligence, may be the true limiting factor in self-improvement loops. Image from Wikimedia Commons

Beneath the headline warnings about recursive self-improvement lies a critical acknowledgment: compute capacity is the binding constraint in every scenario Anthropic modeled. The report names chip fabrication, grid expansion, and interconnect bandwidth as the factors most likely to cap progress ahead of raw intelligence gains.

These limits are far from theoretical. SK hynix and Micron have sold out their HBM (High Bandwidth Memory) output for the entire year. High-power transformers carry three-to-five-year lead times, switchgear is booked into 2028, and grid-interconnection queues now run three to seven years in many regions. A Sightline Climate analysis estimates that 30% to 50% of large data centers due to open in 2026 will slip or cancel entirely.

U.S. data centers consumed about 4.4% of national electricity in 2023 — a share the Department of Energy's Lawrence Berkeley National Laboratory expects to reach 6.7% to 12% by 2028. Meanwhile, the four largest hyperscalers are on track to spend more than $650 billion on AI infrastructure this year alone.

Not Everyone Is Convinced

Skepticism about Anthropic's framing has come from multiple directions. Cognitive scientist Gary Marcus called the piece a "bait and switch" on his Substack, arguing the company had shown faster coding under human direction rather than a system genuinely improving itself. Bentley University mathematician Noah Giansiracusa told Scientific American bluntly, "I don't think it's a genuine call to slow down."

The timing certainly invites scrutiny. Anthropic's report arrived just days after the company confidentially filed for an IPO at a reported valuation near $965 billion — a juxtaposition that positions the company as a front-runner lobbying for rules it would help set. In April, Anthropic claimed its Mythos Preview model had found "thousands" of severe zero-day vulnerabilities, a statement that later drew scrutiny over the size of the underlying manual review sample.

Anthropic is also far from alone in seeing rapid AI-generated code growth. Google CEO Sundar Pichai said in an April blog post that 75% of new code at Google is now AI-generated and approved by engineers, up from 50% the previous autumn. OpenAI's Codex agent is described internally as "a very early version of an AI researcher," and Chinese developer MiniMax marketed its M2.7 model in March as self-evolving.

AI robot and human hands
The AI industry is divided on whether self-improvement poses an immediate risk or is being used as a negotiating tactic. Photo by Igor Omilaev on Unsplash

What Independent Measurements Show

Independent benchmarks provide some grounding. METR found last year that the length of task an AI can complete with 50% reliability has been doubling roughly every seven months. On its RE-Bench research benchmark, the best AI agents beat human experts given two hours, but humans pulled ahead at eight hours and roughly doubled the top agent's score at 32 hours. The data suggests AI's current advantage sits in short, well-defined bursts — not the sustained, open-ended work that genuine research depends on.

The debate over whether compute bottlenecks will ultimately constrain any out-of-control self-improvement loop remains unresolved. Forethought researcher Tom Davidson argues there's a chance that compute constraints won't "slow down a software intelligence explosion until its late stages." Epoch AI counters that if compute and cognitive labor are complements rather than substitutes, software-only acceleration stalls once it hits a compute wall.

No One Is Actually Going to Pause

Anthropic's own position on halting development reveals the practical reality. The company says it will only pause if rival labs at or near the frontier do the same in a verifiable way — a condition that, given the current AI arms race dynamics, effectively means never. Every lab that's this far down the road is racing, not coasting, and Anthropic's report reads as much as a demonstration of capability as it does a warning about safety.

The throughline is clear: the real governor on AI progress isn't safety concerns or regulatory hesitation — it's the physical and economic constraints of the global compute supply chain. Whether self-improving models become a transformative force or a bounded phenomenon depends less on the algorithms themselves and more on how many data centers the world can actually build, power, and connect.

Related Articles

Frontier Airlines site leaks all personal info with just a glance at a boarding pass, researcher claims — booking number and last name nets you every passenger's personal info, including address, passport, TSA PreCheck, and most credit card info
News4 min

Frontier Airlines site leaks all personal info with just a glance at a boarding pass, researcher claims — booking number and last name nets you every passenger's personal info, including address, passport, TSA PreCheck, and most credit card info

Security researcher Bob reveals Frontier Airlines exposed passenger passport data, credit card info, and TSA PreCheck numbers through a simple API flaw — just a boarding pass code and last name.

Hardware
Kaspersky finds malware hidden in Steam Wallpaper Engine that hijacks accounts to spread itself — dozens of malicious packages downloaded tens of thousands of times
News3 min

Kaspersky finds malware hidden in Steam Wallpaper Engine that hijacks accounts to spread itself — dozens of malicious packages downloaded tens of thousands of times

Kaspersky researchers uncover a months-long malware campaign exploiting Steam's Wallpaper Engine executable wallpaper feature, with dozens of malicious packages downloaded tens of thousands of times to deliver infostealers and ransomware.

Hardware
Chinese memory brands ditch Samsung and Micron for homegrown CXMT and YMTC silicon — Corsair, HP, and Dell are already adopting the China-produced DDR5 chips
News4 min

Chinese memory brands ditch Samsung and Micron for homegrown CXMT and YMTC silicon — Corsair, HP, and Dell are already adopting the China-produced DDR5 chips

Chinese memory brands Gloway and KingBank ditch Samsung and Micron for homegrown CXMT DDR5 chips, with Corsair, HP, and Dell already adopting the China-produced memory modules.

Hardware
AMD silently removes memory encryption from consumer Ryzen CPUs, leaving users unaware that they may be vulnerable — security feature vanishes after newer AGESA firmware, AMD engineers go radio silent when pressed about the change
News3 min

AMD silently removes memory encryption from consumer Ryzen CPUs, leaving users unaware that they may be vulnerable — security feature vanishes after newer AGESA firmware, AMD engineers go radio silent when pressed about the change

AMD quietly disabled Transparent Secure Memory Encryption (TSME) on consumer Ryzen CPUs through AGESA 1.2.7.0 firmware updates. AMD engineers went silent when pressed for answers.

Hardware