AI Isn’t Breaking Cybersecurity. It’s Breaking Trust.

This is the central warning running through Leopold Aschenbrenner’s Situational Awareness: The Decade Ahead. Written by a former member of OpenAI’s superalignment team, it has quietly become required reading for frontier‑lab strategists, infrastructure investors, and senior technologists – from Nat Friedman and Daniel Gross to the Collison brothers at Stripe. When that crowd aligns on something, it usually means the ground is shifting.

Aschenbrenner’s argument is blunt: AI capability is compounding at a rate that outpaces institutional response; compute, energy, semiconductors, and data centres are becoming strategic national assets; and the next decade will be defined by discontinuities, not linear progress. He argues that Artificial General Intelligence could plausibly emerge by 2027. I’m not interested in debating the exact timeline – the direction of travel is clear, and the first casualty is trust.

How Trust Collapses

To understand Aschenbrenner’s warning about compounding capability, we have to look at how it manifests practically. The physical world functions on structural buffers, but cybersecurity was built for a human‑speed world. Sophisticated attacks once required skilled operators, time, infrastructure, and patience. That friction acted as a natural trust signal: if something took time and effort, it was more likely to be genuine.

AI is erasing those signals.

Phishing emails that once required an afternoon for a native speaker now take seconds to generate at scale. Reconnaissance that demanded coordinated teams can be automated globally. Social engineering that relied on craft and patience is now accessible to anyone with a browser. What looked like isolated experiments a few years ago has now become the baseline for modern intrusions – and each one is ultimately a trust failure.

The Social Layer – Trust in People Fails

In the Hong Kong case, a finance employee authorised a multimillion‑dollar transfer after joining a video call where every participant including the “CFO” was an AI‑generated deepfake. The attackers did not need to bypass an infrastructure perimeter – they breached trust in human perception.

The Identity Layer – Trust in Credentials Fails

Attackers increasingly bypass passwords and Multi-Factor Authentication (MFA) by stealing and replaying authentication tokens. The Storm‑0558 incident showed how forged Microsoft tokens were used to access government emails. Stripe’s2026 experience with automated agent fraud and API token‑draining attack also demonstrated how industrialised this has become.

Identity systems only work if tokens, sessions, and approvals can be trusted. AI‑accelerated impersonation breaks that assumption.

The Infrastructure Layer – Trust in Systems Fails

In a recent case, an intruder used AI‑driven tooling to map a complex AWS environment, identify misconfigurations, and execute a full compromise in eight minutes – a workflow that previously required days of expert knowledge.

CrowdStrike’s 2026 Global Threat Report puts the average breakout time at just 29 minutes, with the fastest observed instance at 27 seconds. Other vendors claim response times in seconds, collapsing timelines that once demanded hours of human triage. This hyper-acceleration is the algorithmic compounding Aschenbrenner warned about taking place in real time. The battle is no longer human vs. machine; it is algorithm vs. algorithm.

Identity and Authenticity Failure – The Perimeter has Collapsed

For years, organisations have repeated the mantra that “identity is the new perimeter.” But identity only works if it can be trusted. In a world of synthetic media, behavioural mimicry, and automated impersonation, identity is no longer a reliable control. If I cannot trust the voice on the call, the face on the video, or the approval in the email, the question stops being “who has access?” and becomes “what is real?”

Identity is no longer a security control – it is now a contested space.

This is not a technical issue to be delegated downward; it is a governance failure in the making.

So far, every failure described here has been attacker vs defender. But an even more fragile dependency sits above them: trust in the institutions that control AI itself – providers and governments who can revoke access overnight.

The Geopolitical Pivot: The Fable 5 Shock

The first major trust failure of the AI era did not come from a malicious threat actor. It came from the institutions meant to protect us – proving that trust can be withdrawn at a national scale.

In June 2026, Anthropic was forced to abruptly disable access to its most advanced models, Fable 5 and Mythos 5, for all users worldwide after the U.S. government issued an export‑control directive citing national security concerns.  The order barred any foreign national,  including Anthropic’s own employees, from accessing the models, leaving the company no choice but to shut them down entirely.

This was not a cyber incident. It was a governance reality check and a direct validation of Aschenbrenner’s macro thesis. The infrastructure underpinning AI, including compute, energy, semiconductors, and cloud concentration is no longer just a collection of technology dependencies. These are strategic national assets and geopolitical levers, and therefore national targets.

Whether or not access is ultimately restored, the episode has proved something leaders cannot unlearn: access to frontier AI can be withdrawn by a single government directive, with little notice and zero recourse.

Yet most organisations still approach AI through a narrow productivity lens. Few are asking the harder question: What if our ability to govern this technology is already lagging behind our ability to deploy it?

The gap between what organisations build and what they can control and protect is where failures start. In the AI era, organisational agility is not a cultural aspiration; it is a trust control.

Trust is the New Battlefield

Business and cybersecurity leaders face a definitive choice: continue optimising for a world of gradual evolution that no longer exists, or begin preparing for a world in which identity, authenticity, and trust are contested at scale.

The next decade of cyber risk won’t be about protecting systems – it will be about protecting trust. Trust in identity, trust in data, trust in infrastructure, and trust in what is real.

Note: I wrote this. AI helped me fact-check it, it was peer-reviewed/challenged by SME friends I trust.  In an essay arguing that AI is eroding our ability to trust what is real, I thought you would want to know that too.