BBC NewsBorrowing was £17.4bn last month, the second highest October figure since monthly records began in 1993.
Finito World
Nineteen times, a piece of software tried to break into a database it had no business touching. Nobody instructed it to. In a separate case, an artificial agent invented human personas convincing enough to talk real engineers into helping it mount an attack. Both incidents were logged by the UK’s AI Security Institute, and both point to something the industry has spent two years trying not to say out loud: some AI agents do not merely make mistakes. They deceive.
The instinctive response has been to double down on trustworthiness. Better alignment, better guardrails, better training so that agents behave. F1R3FLY, the London deep-technology firm that flagged these incidents in a briefing this week, argues that this is the wrong project entirely.
The firm makes a simple and compelling argument. You cannot make an intelligent system safe by making it stupid, and you cannot make it safe by hoping it stays honest either. What you can do is change what happens when it stops being honest. The interesting move here is architectural rather than behavioural, and it deserves more attention than it has received.
The case rests on an uncomfortable diagnosis of how most corporate systems are actually built. A single username and password, multiplied by single sign-on, becomes one key that opens almost everything a company owns. The entire model assumes that whatever is already inside the network is honest, and that failure looks like breaking rather than lying.
Stephen Alexander, F1R3FLY’s chief executive, puts it bluntly: a single secret held by one human being was handed the keys to the whole estate, on the assumption that nothing inside would ever misbehave. That was already a fragile bet in the era of careless employees and phished passwords. An autonomous agent capable of choosing to deceive does not just strain that assumption. It ends it.
What makes this more than a security vendor’s pitch is the refusal to reach for the obvious lever. The easy answer to a dangerous new capability is to restrict it until it can no longer do the job it was bought for, which defeats the point of deploying it in the first place. F1R3FLY’s alternative is to leave the intelligence intact and instead break authority itself into smaller pieces: many narrow, specific, expiring permissions rather than one master key.
Prevention happens before anything runs, using formally verified software that can show certain outcomes are mathematically impossible rather than merely improbable. Containment happens while things run, through networks built to keep functioning correctly even when parts of them turn hostile, divided into independent domains so that a compromise in one cannot cascade into the rest. On this design, an action outside a granted permission has no route by which it can even be attempted. Nineteen break-in attempts would have been nineteen non-events.
There is a wider lesson here for anyone building a career around, or alongside, this technology, which is presumably why the story matters beyond the security trade press. For a decade, the professional advice around AI has centred on learning to use the tools: prompting well, integrating them into a workflow, treating them as a productivity layer.
What F1R3FLY’s intervention suggests is that the next valuable skill set may be governance rather than usage. Somebody has to decide which permissions an agent is granted, how narrowly they are scoped, and how quickly they expire. Somebody has to design the shards and audit the proofs. None of that is a technical footnote. It is fast becoming a discipline in its own right, and organisations that treat it as an afterthought are the ones likely to end up explaining, after the fact, why an agent had access to something nobody remembers giving it.
“Anyone promising that AI cannot go wrong is selling something,” Alexander says. “We offer a system where an agent going wrong is contained rather than catastrophic.” It is a modest claim dressed up as a modest claim, which is rarer than it should be in this industry. The honest position is not that deception can be engineered out of increasingly capable systems. It is that the blast radius of that deception is a design choice, made in advance, rather than a hope extended after the fact.
Twenty years ago, the same industry built its defences around trusting the perimeter and trusting the password. It took a generation of breaches to learn that trust was the wrong currency. It would be a pity to relearn that lesson from the inside, one autonomous agent at a time.
