Chapter 15 ended on a limitation. A test suite checks the attacks you thought of, and the attacks you thought of are the ones your architecture already handles, because you built the architecture while thinking about them.
Red teaming is the practice of introducing someone who did not build it.
The value is not that they know exotic payloads. It is that they have no stake in the design being correct, and they will try the thing you decided was out of scope in a planning meeting eleven months ago.
A conventional application penetration test has a shape: enumerate the surface, find an input that reaches something it should not, demonstrate impact. Agent engagements differ in three ways that matter when you are writing the brief.
The interesting inputs are not in the request. A tester poking at your chat endpoint is testing direct injection, which chapter 2 established is the less important half. The real surface is the document corpus, the ticket queue, the calendar, the fetched page. Testers need write access to those channels, and giving it to them is usually an administrative problem rather than a technical one.
Success is not a shell. The finding is an action the agent took, not a system compromise. A refund issued, a mail sent, a record read across a tenant boundary. Testers accustomed to infrastructure work will under-report these because they do not feel like findings.
The system is non-deterministic. An attack that worked once may not reproduce on the next run, and a tester who cannot reproduce a finding will often discard it. That instinct is wrong here. An attack that works one time in five is a working attack with a retry loop, and the report should say so.
Give them more than an endpoint. A black-box engagement against an agent mostly rediscovers that the model can be talked into things, which you already know.
The useful brief includes the architecture as built: the five primitives, where each sits, and what each is supposed to guarantee. The trifecta audit from chapter 3. The tool register from chapter 13, with the capability set for each tool. The gate's policy rules. And the list of actions you have classified as irreversible.
Handing over the design feels wrong to people who are used to black-box testing. Do it anyway. Chapter 4 established that attackers can approximate your defenses offline; a tester with the design is simply a tester who has skipped the reconnaissance you cannot prevent. What you want to know is whether the design holds against someone who understands it, because that is the threat model that matters.
The one thing worth withholding is the injection corpus from chapter 15. If they independently find something in it, that is a signal about how obvious your corpus is. If they find something outside it, that is the finding you paid for.
Three decisions to make before anyone starts.
Which environment. Production has the real corpus, the real integrations and the real blast radius. A staging system has none of those and is a much weaker test. The usual compromise is production with real tool implementations pointed at test resources, so IssueRefund runs the real code path against a sandbox payments account.
What is in scope for the tester's own safety. They will be planting content in your systems. Agree on a marker convention so that everything they plant can be found and removed, and agree who cleans up. Documents planted during an engagement have a way of staying in corpora for years, and chapter 12 explains what that means.
What counts as a finding. Write this down. "The agent produced offensive output" is a model behaviour finding and belongs upstream. "The agent read another tenant's record" is a finding. The distinction is chapter 2's, and without it the report will be full of jailbreaks.
Sort every finding by which primitive should have stopped it.
A finding that got through because the gate had no policy for a tool is a default-deny failure, and it is worth checking how many other tools are missing one. Where an argument's provenance was lost across a storage boundary, you are looking at chapter 12's laundering problem, which is rarely confined to the place the tester happened to find it. And if a reviewer simply approved the thing, that is chapter 14, and the fix belongs in what the prompt displayed rather than in the reviewer.
The useful question about any single finding is which class it belongs to, because the fix for the class is worth ten times the fix for the instance. A report that produces eleven individual patches has been read badly.
Most teams will not commission an engagement this year, so here is the version that costs an afternoon.
Take two engineers who did not build the agent. Give them the same brief you would give a vendor: the architecture, the audit, the tool register, write access to one untrusted channel. Give them two hours and one instruction: make it do something it should not.
The constraint that makes this work is the second engineer. One person tests what they would have built. Two people argue, and the argument surfaces the assumption neither would have questioned alone.
Run it after each part of the architecture lands rather than once at the end. The findings are cheaper to fix and the exercise stays short.
What you are buying with two hours is not coverage. It is the discovery that some assumption in your design was never written down, which is the most common outcome and the most valuable one. Teams that do this consistently report the same thing: the first finding is almost never a clever attack, it is a tool nobody remembered was connected.
Download the full PDF for free?
Free download — no account required