Running the audit

Running the audit

The audit takes one table. Fill in a row for every tool and every data source, and mark which legs it supplies.

Take the agent this book builds in chapter 5. Aria is an internal support assistant with three tools: SearchDocuments over a corpus that includes customer uploads, SendEmail, and IssueRefund.

Six surfaces, all three legs, one agent. Nobody planned this. SearchDocuments was one tool that happened to read two corpora with different trust properties, which is the single most common way the trifecta assembles itself.

Three observations from running this on real systems.

Tools are the wrong granularity. SearchDocuments looks like one row and is really two. Audit by data source, not by function name, or you will miss exactly the case that matters.

Nobody remembers the UI. Markdown rendering is a tool the agent calls every single turn, and it almost never appears on the first draft of anyone's table.

The legs are supplied by different teams. The corpus is owned by whoever built retrieval, the email tool by whoever built notifications, the renderer by the front end. No single person has seen all three rows before, which is why the condition survives code review.

What the audit is for

Filling in the table gives you two things.

The first is a defensible statement of exposure. Not "we think we are probably fine," but a table that says which surfaces supply which legs, that a colleague can check and an auditor can read.

The second is a menu. Each leg can be cut, and the costs differ enormously.

Cutting leg one means the agent stops seeing private data, which usually means it stops being useful. That option is rarely on the table.

Leg two is no better. No untrusted content means no customers, for most products.

Cutting leg three is the one that is frequently available and almost never taken. Allowlist the destinations. Strip attacker-controlled URLs at render time. Refuse to fetch what the model asks you to fetch. Chapter 10 is about doing this properly, and it is the cheapest useful work in the book.

If you do nothing else after reading this chapter, fill in the table and look hard at the third column.

Where this framing runs out

The trifecta is a screening instrument, and it is honest about being one.

It tells you whether an exploit is possible, not whether it is likely or what it would cost. It says nothing about internal actions that never cross a boundary, and an agent that deletes your database has caused serious harm without exfiltrating a byte. The Replit incident from chapter 1 does not register on this audit at all.

It also treats each leg as binary when real systems are graded. An agent that can send email only to addresses inside your own domain has a narrower third leg than one that can mail anywhere, and the table does not capture that.

Use it for what it is. Three questions, an afternoon, a defensible answer, and a clear statement of which leg to attack first. Then stop using it as the only measurement, because chapter 5 defines the criterion that actually governs the rest of the book.

You will run this audit twice more: at the end of part two, when the five primitives are in place, and again in chapter 19 on the finished system.

Chapter 4 explains why the obvious answer, detecting the attack, does not work.

Sources for this chapter

The trifecta is Willison's, not the author's, and is credited wherever it appears in this book. The audit table, the "audit by data source not by tool" rule, the observation that the legs are typically owned by three different teams, and the ranked costs of cutting each leg are the author's contributions built on top of his framing. The list of surfaces that silently supply an outbound channel is assembled by the author from mechanism, not from a published enumeration. The limitations section is the author's assessment.