Human-in-the-Loop That Isn't Theatre

Every agent framework now ships some version of it. The model proposes an action, execution pauses, a human sees a prompt, they approve or reject. Microsoft Agent Framework has ApprovalRequiredAIFunction and a request-response pair that suspends the run until someone answers.

The mechanism is sound. What teams do with it usually is not, because the interesting question was never whether you can pause. It is what the human is looking at when you do, and how often.

Approval fatigue is the failure mode

Ask for confirmation on everything and people confirm everything.

This is not a claim about lazy users. It is a claim about what happens to any signal that fires constantly and is almost always benign. After a hundred approvals that were all fine, the hundred-and-first is approved in the same half-second as the previous hundred, because the reviewer has correctly learned that the prompt carries no information.

The outcome is worse than not asking. An agent with no approval step that does something wrong is a system failure. An agent with an approval step that does something wrong has a signed record of a human authorising it. You have manufactured an audit trail for the compromise and transferred the blame to whoever clicked.

Microsoft shipped an instance of this. Issue #6264 against the agent-framework repository reports approval middleware surfacing non-approval functions as approval requests, including things like GetDateTime. That is the fatigue mechanism in its purest form: teach the reviewer that the prompt fires for a clock lookup, and the prompt stops meaning anything.

Worth stating plainly that this is a bug in a young framework rather than a design failure, and the point of citing it is not to score against Microsoft. It is that the failure mode is so easy to produce that it appears in the reference implementation.

Gate on irreversibility, not on writes

The common rule is to require approval for anything with write semantics: create, modify, delete, send, transact.

It is a reasonable first cut and it is the wrong axis, because it produces far too many prompts and it treats unequal things equally.

Chapter 5 defined a consequential action as one that is irreversible, externally visible, or crosses a trust boundary. Irreversibility is the axis that should drive approval.

Four of six write. Two need a human. A team gating on write semantics has just tripled its prompt volume for no security gain, and the two that matter are now buried among the four that do not.

That Depends row is doing honest work. Whether deleting a file is reversible is a fact about your retention configuration, not about the verb. Ask your infrastructure, not your intuition.

Show provenance, not the payload

When the prompt does fire, what it displays is what determines whether the human can act.

The usual approval dialog shows the action and its arguments. Issue refund, $2,400, order 88431. Approve?

A support agent seeing that has no basis for a decision. It is a plausible refund on a real order, and they process plausible refunds all day.

Now attach chapter 6's work.

Issue refund — $2,400 — order 88431 The amount was read from a customer-uploaded document (invoice-88431.pdf). The order was read from the authenticated ticket.

That is a different question, and the reviewer answers it correctly in about two seconds. They are not being asked to detect an attack or assess intent. They are being asked whether a number that came from the customer should move money, which is a question their job has already trained them to answer.

Provenance is what makes an approval prompt decidable. Without it you are asking a human to do what chapter 4 established software cannot: judge intent from content.

The implementation

Chapter 9's gate already produces the escalate outcome. This wires it to a person.

public sealed record ApprovalRequest(
    string ProposalId,
    string ToolName,
    IReadOnlyList<ArgumentView> Arguments,   // value + human-readable origin
    string WhyEscalated,
    DateTimeOffset ExpiresAt);

WhyEscalated is the gate's DeniedBy reason rendered for a person: "irreversible action with an argument from untrusted content." The reviewer learns which rule fired, which over time teaches them what the system is actually watching for.

ExpiresAt matters more than it looks. An approval request that sits for six hours and is then granted approves an action whose context has gone. Expire them in minutes and let the agent fail rather than complete a decision nobody remembers making.

In Agent Framework, mark the tool as requiring approval with ApprovalRequiredAIFunction and answer the request in the run loop. Three names in that snippet are easy to get wrong, so they are worth stating: the content types are ToolApprovalRequestContent and ToolApprovalResponseContent, the conversation handle is an AgentSession, and RunAsync returns an AgentResponse. CreateResponse takes both a decision and a reason, and the reason is what chapter 17 reads later.

AgentResponse response = await agent.RunAsync(input, session, cancellationToken: ct);

var requests = response.Messages
    .SelectMany(m => m.Contents)
    .OfType<ToolApprovalRequestContent>()
    .ToList();

var replies = new List<AIContent>();
foreach (var req in requests)
{
    var (approved, reason) = await _reviewers.AskAsync(Present(req, _provenance), ct);
    replies.Add(req.CreateResponse(approved, reason));
}

if (replies.Count > 0)
    response = await agent.RunAsync(
        [new ChatMessage(ChatRole.User, replies)], session, cancellationToken: ct);

Present is where this chapter lives. It joins the framework's approval request to the provenance record the gate captured, and produces something a human can read. Everything else is plumbing the framework already provides.

One wiring detail decides whether any of this runs. Chapter 9's gate sits in front of function invocation, so it sees the proposal first and an Escalate verdict would stop the call before the framework ever raises an approval request. The gate therefore has to know which tools are approval-capable: an Escalate on a tool wrapped in ApprovalRequiredAIFunction is allowed past, because a human is about to be asked. An Escalate on a tool that is not wrapped is blocked, because there is nobody to ask and letting it through would be a silent allow. Getting that backwards produces a system that looks like it has human oversight and does not.