Home

/

Prompt Injection: Blast Radius

/

What it actually took

What it actually took

Chapter 19
Part V
4
min read

What it actually took

Worth being concrete about effort, because the abstract version of this work sounds larger than it is.

PrimitiveCodeWhere the time went
Egress~200 linesBuilding the allowlist from real traffic, then a week of report-only
Provenance~150 lines, plus signature churnThreading Tagged<T> through the tool layer
The gate~300 linesWriting the policies, and arguing about which actions are irreversible
Capability~250 linesDeclaring what each task type needs, which nobody knew
Quarantine~200 linesDesigning schemas narrow enough to be worth having

Under 1,200 lines of unremarkable C#. In every row the code was the small part and the decisions were the large one.

That pattern is worth expecting. The gate is three hundred lines and a fortnight, because the fortnight is spent getting a team to agree which actions cannot be undone. The capability issuer is straightforward and the list of operations per task type does not exist anywhere and has to be written by someone who understands the product.

If a plan for this work budgets engineering days and no decision days, it is wrong in a way that will show up in week three.

The order matters more than the total

Ship these in the wrong order and you will spend months before anything improves.

Egress first, because it is independent of everything else and stops the most published attacks. Provenance second, because the gate cannot ask its most useful question without it. The gate third. Capability fourth, since the gate works with a coarse capability model and gets better with a fine one. Quarantine last, because it is the most invasive and the least urgent once a gate is standing.

A team that starts with quarantine, which is the intellectually interesting one, will spend six weeks on schemas while the exfiltration leg stays open.

Where this leaves you

An agent that can be injected, will be injected, and cannot do much about it.

That is the outcome. Not an agent that resists attack, because chapter 4 established you cannot buy that. An agent whose compromise is uninteresting, because the authority to do damage was never sitting where the compromise happens.

The work was ordinary. Type wrappers, a policy engine, scoped tokens, a renderer, an approval screen, some tests. No research, no novel cryptography, nothing that did not exist in 1990. The difficulty was never technical. It was that the industry spent three years asking how to make the model trustworthy, and the answer was to stop needing it to be.

Running this well is ongoing work rather than a project that finishes. The policies drift, the tool register goes stale, the allowlist needs maintaining, and the test corpus grows with each engagement. That is the shape of the commitment, and it is the same shape as the rest of your production surface.

If you would rather have someone build and maintain this alongside your team, that is what You Source does through Dev on Demand: senior engineers delivering task-based work, starting with a Proof of Quality, which is one real task delivered and evaluated before any subscription. The architecture in this book is not proprietary and you do not need us to implement it. The offer is the maintenance.

Chapter 20 is about what none of this fixes.

Sources for this chapter

ClaimSourceStatus
ApprovalRequiredAIFunction, middleware ordering before UseFunctionInvocationhttps://learn.microsoft.com/en-us/agent-framework/agents/tools/tool-approval · https://learn.microsoft.com/en-us/agent-framework/agents/middleware/PRIMARY
The lethal trifecta framing used in the re-run auditWillison, 16 Jun 2025, https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/PRIMARY

Every attack in the "where each attack dies" section is a construction against the reference agent in this book, not a report of a live system, and the outcomes describe what the architecture is designed to do rather than measured results. No claim is made that these seven attacks are representative of what an adversary will try, and chapter 20 addresses what the design does not cover. The checklist ordering is the author's judgement about effort versus reduction, not a measured ranking.

the-three-year-bug
why-the-industry-shipped-anyway
injection-is-not-jailbreaking
why-the-confusion-persists
the-lethal-trifecta
running-the-audit
why-filtering-fails
measured-here-on-a-named-model
why-this-is-structural
what-solved-would-look-like
the-harness
provenance-every-value-knows-where-it-came-from
on-the-reference-agent
quarantine-the-planner-never-reads-the-mail
what-two-models-cost-in-practice
capability-authority-the-agent-cannot-widen
expiry-is-a-feature
the-gate-the-model-proposes-code-disposes
the-policy
failing-closed
egress-closing-the-exfiltration-leg
how-much-can-actually-leak
sandboxing-containing-the-code-the-agent-writes
the-sandbox-held-and-it-did-not-help
poisoned-memory-poisoned-retrieval
cleaning-up-afterwards
the-tool-supply-chain
mcp-and-the-rest
human-in-the-loop-that-isnt-theatre
when-there-is-nobody-there
testing-for-injection
measuring-coverage-not-pass-rate
red-teaming-agents
a-finding-worked-through
when-it-happens-anyway
what-the-logs-cost-you-in-an-incident-you-did-not-have
governance-procurement-and-the-regulator
writing-the-policy
end-to-end
what-it-actually-took
what-stays-broken
why-this-is-probably-structural
the-trifecta-audit-worksheet
action-schema-and-policy-reference
control-mapping
prompt-injection-sources
incidents
appendix-e-what-we-re-ran-ourselves
e4-the-control-that-keeps-e2-and-e3-honest

Download the full PDF for free?

Free download — no account required

Get the PDF
Get the PDF
Related Chapters
Free Download
Get the full PDF
All pages, including all code examples, diagrams, and the appendix reference card.
No spam. Unsubscribe at any time.
Your email won't be shared.
Oops! There's a problem with your request. We're working on fixing it. Please try again later.