Four years after the vulnerability was named, the person who named it still says we do not know how to reliably prevent it. That is not a gap in the products. It follows from what a language model is.
A model takes a sequence of tokens and predicts the next one. Instructions and data are the same tokens. There is no channel separation because there is no channel, and the property that makes these systems useful over arbitrary text is the same property that makes them unable to refuse arbitrary text.
Better models may narrow this. Instruction hierarchies have improved and will keep improving. But a system whose safety rests on a model's judgement is a system whose safety is a probability, and an attacker gets unlimited attempts at a probability.
So the reasonable expectation is that this does not get solved in the sense of going away. It gets managed, the way memory safety was managed for thirty years before the industry changed languages, and the way SQL injection is still managed in code written last month.
That makes containment a permanent discipline rather than a stopgap. The architecture in this book is not a bridge to a future where models can be trusted. It is what building on models looks like.
Some of it. That is the nature of writing about a field moving this fast, and it is worth saying which parts are most likely.
The implementation details will date first. The framework APIs in these pages are from a product that reached 1.0 five months before publication, and two of the GitHub issues cited in chapter 14 may be closed by the time you read it.
The measurements will date next. Chapter 4's evasion evidence is from April 2025, and guardrail products have shipped since. The structural argument does not depend on those numbers, which is why the chapter was built to survive them, but a reader quoting the specific figures in 2028 should check them first.
What should last is the reasoning: that a control a model can be argued out of is not a control, that provenance is the property no classifier can supply, and that the decision belongs in code. If those turn out to be wrong, the book is wrong in the way that matters and the implementation details will be the least of it.
If everything else here is forgotten, keep this.
If the enforcement lives in the prompt, it is advice.
The Replit agent read a code freeze and destroyed a production database. The freeze was real, the instruction was clear, the model understood it, and none of that mattered, because nothing in the execution path enforced anything.
Every mechanism in this book is one application of that single idea: move the decision out of the model and into code that cannot be argued with.
That is the whole thing. Everything else is implementation.
| Claim | Source | Status |
|---|---|---|
| "We still don't know how to 100% reliably prevent this from happening"; LLMs follow instructions in content | Willison, 16 Jun 2025, https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ | PRIMARY |
| Guardrail evasion against six production systems, including via offline white-box approximation | Hackett et al., arXiv 2504.11168, https://arxiv.org/abs/2504.11168 | PRIMARY |
| Replit agent destroyed production data during an active code freeze | AI Incident Database, Incident 1152, https://incidentdatabase.ai/cite/1152/ | PRIMARY (register) |
| Open problems in multi-agent security | Open Challenges in Multi-Agent Security, arXiv 2505.02077, https://arxiv.org/pdf/2505.02077 | PRIMARY |
The argument that prompt injection is structural rather than a defect awaiting a patch is the author's, built on Willison's position and the evasion evidence, and it is a prediction rather than a finding. The comparison to memory safety and SQL injection is illustrative. The advice on proportionality for small teams, the treatment of other agents as untrusted services, and the assessment that multi-agent provenance is unsolved are the author's judgements as of September 2026.
Download the full PDF for free?
Free download — no account required