The Gateway

Why a chokepoint

Chapter 12's controls are excellent and they are distributed. Every developer, every feature, every code path has to get them right, and the number of code paths only goes up.

A gateway inverts that. Model calls go through one component. That component applies policy. A new feature gets the controls by default, and a developer who wants to bypass them has to work at it visibly rather than accidentally.

It also gives you three things that are hard to retrofit: a single place to change policy without redeploying every service, a single place where every call is observable, and a single answer to "what controls were in force on 3 March?" That last one is Chapter 17's entire subject, and it is almost impossible to produce from scattered per-service logic.

The shape in ASP.NET Core

The gateway is a small service. Applications call it instead of calling the provider.

public sealed class ModelGateway(
    IDetectionPipeline detection,
    IPolicyStore policies,
    IModelProvider provider,
    ILogger<ModelGateway> logger)
{
    public async Task<GatewayResult> CompleteAsync(
        GatewayRequest request, CancellationToken ct)
    {
        var policy = await policies.ResolveAsync(
            request.Tenant, request.Route, ct);

        var inbound = await detection.ApplyAsync(
            request.Prompt, policy.Inbound, ct);

        var completion = await provider.CompleteAsync(
            inbound.Text, request.Options, ct);

        var outbound = await detection.ApplyAsync(
            completion, policy.Outbound, ct);

        return new GatewayResult(
            Text: outbound.Text,
            PolicyVersion: policy.Version,
            InboundFindings: inbound.Findings.Count,
            OutboundFindings: outbound.Findings.Count);
    }
}

Four things in that method are the chapter.

Policy resolved per tenant and per route

A single global policy forces a bad compromise. The route that summarises internal changelogs and the route that processes patient correspondence do not need the same safeguard, and applying the strict policy everywhere degrades features that did not need it, which is how people end up routing around the gateway.

public sealed record DetectionPolicy(
    string Version,
    IReadOnlyList<string> Entities,
    double ScoreThreshold,
    SafeguardKind Safeguard,
    FailureMode OnDetectorUnavailable);

public sealed record GatewayPolicy(
    string Version,
    DetectionPolicy Inbound,
    DetectionPolicy Outbound);

Policies live in configuration, versioned in source control, deployed independently of application code. Tightening a threshold becomes a reviewed pull request against a policy file rather than a change to six services.

Give every policy an immutable version string. Never edit a policy in place; publish a new version. The version is what makes the audit trail meaningful.

Both directions

The outbound pass is the one teams forget, and it closes door five from Chapter 3.

Three things can be in a completion that were not in the prompt. The model can regurgitate data that retrieval pulled in, which is Chapter 15's problem arriving at your door. It can infer something about a person, and an inference about an identified person is personal data you generated. And it can carry rendered exfiltration, which is what EchoLeak exploited: markdown that causes a client to fetch an attacker-chosen URL with data in the query string.

The third one is not a PII detection problem and should not be handled by the detector. It is a rendering problem, and it is handled with an allowlist:

private static readonly Regex MarkdownImage =
    new(@"!\[[^\]]*\]\(([^)]+)\)", RegexOptions.Compiled);

public static string StripUntrustedImages(string markdown, HashSet<string> allowedHosts)
{
    return MarkdownImage.Replace(markdown, m =>
    {
        var url = m.Groups[1].Value;
        return Uri.TryCreate(url, UriKind.Absolute, out var uri)
               && allowedHosts.Contains(uri.Host)
            ? m.Value
            : "[image removed]";
    });
}

Apply the same thinking to links, to HTML if you render any, and to anything else in your output that causes a client to make a request. The rule is that model output is untrusted input to your renderer, because something upstream of the model may have been controlled by someone else.