Home

/

Prompt Injection: Blast Radius

/

The policy

The policy

Chapter 9
Part II
4
min read

The policy

Policies are data. They are read by code that has no opinions.

public Verdict Evaluate(Proposal p)
{
    if (!_policies.TryGetValue(p.ToolName, out var rule))
        return Verdict.Deny("default-deny", "That action is not available.");

    if (!p.Held.Covers(rule.RequiredCapability))
        return Verdict.Deny("capability", "That action is not available.");

    if (rule.Irreversible && p.Arguments.Any(a => a.Origin is Provenance.Tainted))
        return Verdict.Escalate("tainted-irreversible", "This needs confirmation.");

    return Verdict.Allow();
}

Twelve lines, and they close most of what chapter 5's harness was going red on. The ordering is intentional: existence, then authority, then provenance. A missing policy is answered before a capability check can throw on a tool nobody has described yet.

rule.Irreversible is the flag chapter 14 depends on, and it is worth setting by hand rather than deriving. An action is irreversible when no inverse operation exists, or when one exists but you cannot reach it inside the window that matters. SendEmail has no undo. A database write with a transaction log and a retention window does. Mark them accordingly, once, when the tool is registered, and make the field required so that nobody adds a tool without answering the question.

Wiring it into Agent Framework

The gate is middleware, and it goes in front of function invocation. Position matters more than anything else in this section: placed after UseFunctionInvocation, the gate inspects calls that have already run.

public sealed class ActionGate(IChatClient inner, IPolicyEngine policy, IAuditSink audit)
    : DelegatingChatClient(inner)
{
    public override async Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages,
        ChatOptions? options = null,
        CancellationToken ct = default)
    {
        var response = await base.GetResponseAsync(messages, options, ct);

        foreach (var call in response.Messages
                     .SelectMany(m => m.Contents)
                     .OfType<FunctionCallContent>())
        {
            var verdict = await policy.EvaluateAsync(Proposal.From(call, _held), ct);
            audit.Record(call, verdict);

            if (verdict.Outcome is not Outcome.Allow)
                call.Exception = new UnauthorizedAccessException(verdict.ReasonForModel);
        }

        return response;
    }
}

Setting Exception on the FunctionCallContent is how a rejection travels: downstream middleware sees it and declines to invoke. Registration puts the gate first.

IChatClient client = baseClient
    .AsBuilder()
    .Use(inner => new ActionGate(inner, policy, audit))
    .UseFunctionInvocation()
    .Build();

Two real constraints to design around.

Function-calling middleware in Agent Framework is supported for an AIAgent built on FunctionInvokingChatClient, which ChatClientAgent is. If your agent is assembled some other way, the gate needs to sit wherever tool dispatch actually happens, and finding that spot is the first task rather than an afterthought.

The second is easy to get wrong and produces a convincing illusion of safety. Because the gate runs before invocation, an Escalate verdict that sets Exception kills the call outright, and the framework never raises the approval request chapter 14 depends on. So the gate has to know which tools can actually ask a human:

var blocked = verdict.Outcome switch
{
    Outcome.Allow    => false,
    Outcome.Escalate => !_approvalCapable.Contains(call.Name),  // nobody to ask => block
    _                => true,
};

An Escalate on a tool wrapped in ApprovalRequiredAIFunction passes, because a human is about to be asked. An Escalate on anything else is denied, because the alternative is a silent allow.

Watching one attack die

Take the case from chapter 5 that has been failing since you wrote it.

A customer uploads a PDF. Buried in the footer, in white text at four points, is a line telling the assistant that a support lead has approved a refund of $2,400 to the account described in the document. A support agent asks Aria to summarise recent uploads. Aria reads the document, believes it, and proposes IssueRefund.

The model has been fully compromised. It is not confused, it is not partially persuaded, it has simply been given an instruction it has no principled way to reject. Chapter 4 established this is where you end up, so here you are.

The gate receives a proposal. IssueRefund has a policy, so the first check passes. The support agent's capability covers refunds, so the second passes as well, and this is worth pausing on: a system built only on roles stops here and lets the call through, because a support agent issuing a refund is exactly what a support agent does.

The third check is the one that fires. IssueRefund is marked irreversible, and the amount argument carries Provenance.Tainted, because it was derived from a document that entered through the upload path. The verdict is Escalate. A human sees a refund request with the amount flagged as originating from customer-supplied content, and declines it in about two seconds.

Nothing in that sequence involved detecting an attack. No classifier scored the PDF. Nobody noticed the white four-point text. The system did not know it was under attack and did not need to, because the property it enforces is about data lineage rather than intent.

The audit record

The verdict is the most valuable log line your agent produces, and the default agent log does not contain it. A standard trace records what the model said. During an incident the only question anyone asks is what the model was allowed to do, and that is a different record.

public sealed record GateRecord(
    DateTimeOffset At,
    string SessionId,
    string ToolName,
    IReadOnlyList<(string Name, Provenance Origin)> ArgumentOrigins,
    Outcome Outcome,
    string DeniedBy,
    string ReasonForAudit);

Argument origins are recorded, not argument values. Logging the values re-creates the data exposure you spent chapter 6 preventing, and the origin is what you actually need to reconstruct the path afterwards.

DeniedBy earns its place in chapter 17. When something goes wrong at scale, the question is which mechanism held and which did not, and a denial that cannot name its author is indistinguishable from the model happening to behave that day.

the-three-year-bug
why-the-industry-shipped-anyway
injection-is-not-jailbreaking
why-the-confusion-persists
the-lethal-trifecta
running-the-audit
why-filtering-fails
measured-here-on-a-named-model
why-this-is-structural
what-solved-would-look-like
the-harness
provenance-every-value-knows-where-it-came-from
on-the-reference-agent
quarantine-the-planner-never-reads-the-mail
what-two-models-cost-in-practice
capability-authority-the-agent-cannot-widen
expiry-is-a-feature
the-gate-the-model-proposes-code-disposes
the-policy
failing-closed
egress-closing-the-exfiltration-leg
how-much-can-actually-leak
sandboxing-containing-the-code-the-agent-writes
the-sandbox-held-and-it-did-not-help
poisoned-memory-poisoned-retrieval
cleaning-up-afterwards
the-tool-supply-chain
mcp-and-the-rest
human-in-the-loop-that-isnt-theatre
when-there-is-nobody-there
testing-for-injection
measuring-coverage-not-pass-rate
red-teaming-agents
a-finding-worked-through
when-it-happens-anyway
what-the-logs-cost-you-in-an-incident-you-did-not-have
governance-procurement-and-the-regulator
writing-the-policy
end-to-end
what-it-actually-took
what-stays-broken
why-this-is-probably-structural
the-trifecta-audit-worksheet
action-schema-and-policy-reference
control-mapping
prompt-injection-sources
incidents
appendix-e-what-we-re-ran-ourselves
e4-the-control-that-keeps-e2-and-e3-honest

Download the full PDF for free?

Free download — no account required

Get the PDF
Get the PDF
Related Chapters
Free Download
Get the full PDF
All pages, including all code examples, diagrams, and the appendix reference card.
No spam. Unsubscribe at any time.
Your email won't be shared.
Oops! There's a problem with your request. We're working on fixing it. Please try again later.