Failure is a policy decision

Failure is a policy decision

Chapter 9 covered failing closed at the call site. At the gateway it becomes explicit and configurable, which is better, because the right answer differs by route.

public enum FailureMode
{
    Reject,              // return an error; nothing reaches the provider
    DeterministicOnly,   // proceed with Chapter 5's in-process layer only
    Proceed              // requires an explicit, recorded exception
}

catch (DetectorUnavailableException)
{
    switch (policy.Inbound.OnDetectorUnavailable)
    {
        case FailureMode.Reject:
            throw;

        case FailureMode.DeterministicOnly:
            logger.LogWarning(
                "Detector unavailable; degraded to deterministic layer on {Route} under policy {Version}",
                request.Route, policy.Version);
            inbound = _deterministic.Apply(request.Prompt, policy.Inbound);
            break;

        case FailureMode.Proceed:
            logger.LogError(
                "Detector unavailable; proceeding UNFILTERED on {Route} under policy {Version}",
                request.Route, policy.Version);
            inbound = DetectionResult.Unfiltered(request.Prompt);
            break;
    }
}

Proceed exists because some routes genuinely carry no sensitive data and should not have an availability dependency on the detector. Making it a named enum member with an error-level log means choosing it is a decision someone made, recorded in policy, visible in configuration review, and countable in your telemetry. That is very different from a catch block that silently does the same thing.

Count the degradations. A dashboard showing "requests processed in degraded mode" is the difference between knowing your control was off for six hours and finding out during an audit.

What the gateway records

This is the artefact Chapter 17 needs, so define it deliberately.

public sealed record GatewayAuditRecord(
    Guid RequestId,
    string Tenant,
    string Route,
    string PolicyVersion,
    string ProviderModel,
    int InboundFindingCount,
    IReadOnlyDictionary<string, int> InboundFindingsByType,
    int OutboundFindingCount,
    bool Degraded,
    DateTimeOffset At);

Read that list for what is missing. The prompt is not in it, and neither is the completion, the detected values, or the token map. The record describes what happened to a request without reproducing a single thing the request contained.

Counts and types, not content. "Route ticket-summary, policy v14, 3 PERSON and 1 US_SSN found inbound, 0 outbound, not degraded." That record proves the control ran, proves which policy governed it, and is itself harmless. An audit log that contains the personal data it is auditing has reproduced the problem with a longer retention period, and it will be the thing you are asked about.

If you truly need content for debugging, put it behind a time-boxed, per-route, off-by-default flag with its own retention, and treat it as the sensitive store it is.

Do not use an LLM as the detector

It comes up in every design review, so here is the answer.

Using a second model call to find personal data in the first prompt is slower by orders of magnitude, non-deterministic, more expensive per request, and unauditable, because you cannot explain why it did or did not fire on a given input. It also sends the sensitive text to a model, which is what you were trying to avoid, with the additional property that the detection call is now itself subject to prompt injection.

The practitioner consensus across every independent source is consistent: detection is NLP and pattern matching, running deterministically and in your perimeter. Use a model for guardrail classification of intent if you like. Do not use one to find an IBAN.

Where it sits

The gateway should be a separate service rather than a library, for one reason: a library can be bypassed by not referencing it, and a network boundary can be enforced. If your applications can only reach the model provider through the gateway, because egress rules say so, then the control is architectural rather than advisory.

That is worth the operational cost for anything handling regulated data. For a small team with one application, a library with the IPromptSafe marker from Chapter 12 and a well-tested pipeline is a reasonable place to start, as long as you know which property you gave up.

You have now routed every piece of sensitive text in the organisation through two components you operate: a gateway and a detection container. Those components see everything. The next chapter is about what makes them defensible.

Sources for this chapter

• Layered detection, redaction on both request and response, self-hosted redaction driven by versioned policy, keeping the vault outside model reach, and not using a second LLM as the detector — consistent practitioner guidance across independent sources: https://techcommunity.microsoft.com/blog/azuredevcommunityblog/introducing-pii-shield-a-privacy-proxy-for-every-llm-call/4514726 · https://blog.logrocket.com/build-local-ai-proxy-redact-pii-before-llms/ · https://predictionguard.com/blog/pii-detection-redaction-llm-pipelines-regulated-industries · https://www.gravitee.io/blog/how-to-prevent-pii-leaks-in-ai-systems-automated-data-redaction-for-llm-prompt
• Rendered exfiltration via markdown images, and EchoLeak (CVE-2025-32711) — https://wraith.sh/learn/markdown-image-exfiltration
• Immutable audit logs mapped to NIST AI RMF and OWASP — named in the practitioner guidance above.

The policy shape, the three-member FailureMode enum and the content-free audit record are the author's design. The argument that a gateway should be a network boundary rather than a library is a judgement about enforceability.