Choosing the operating point

Choosing the operating point

Now the decision the F1 score was hiding.

A false negative leaks. A false positive annoys. These are not symmetric, and the ratio between them is yours to set rather than the tool's.

Sweep the threshold and look at the shape:

Those figures are illustrative. The shape is not: recall falls away faster than precision rises, which means the cost of the last few points of recall is steep.

Three things should drive where you sit.

Blast radius, from Chapter 1. A prompt to a zero-retention endpoint that vanishes after the call tolerates a lower threshold than text you are about to embed into a vector store forever.

Entity severity. Run different thresholds per entity. A missed CREDIT_CARD and a missed LOCATION are not the same event. Threshold cards and national identifiers aggressively, names moderately, locations loosely.

Whether a human is in the loop. If output goes straight to a customer, false positives are a product defect. If it goes to an internal reviewer who can see the original, they are a minor annoyance and you should be far more aggressive.

Write the choice down with its reasoning. When an auditor asks why the threshold is 0.6, "it was the default" is a poor answer and "we measured precision and recall on 340 labelled production samples and accepted 0.74 recall on PERSON because the output is reviewed by an internal agent before sending" is an excellent one.

What to do with a bad number

You will get a bad number. The benchmarks put cross-domain detection around 0.5, and you have no reason to expect better.

Do not respond by raising the threshold until the number looks good. That changes the measurement, not the risk.

Four responses actually help.

Minimise the input. Every field you stop sending is a set of entities the detector no longer has to catch. This is Chapter 12 and it beats every detector improvement available to you, usually by a large margin, usually in less code.

Add deterministic rules for what you can validate. Chapter 5's layer will not move your F1 much, but it converts your worst-consequence entities from probabilistic to near-certain, and consequence is what you care about.

Add a domain recogniser. If your data contains a customer reference in a known format, Presidio lets you register a pattern recogniser for it. Your own identifiers are exactly the entities no general detector will ever find, and exactly the ones you can detect perfectly.

Design for the miss. Fail closed, filter the output as well as the input, keep the real identifier out of band. That is Part IV, and it is the only response that helps when the detector is simply wrong.

Re-measure, because the gap reopens

The demo gap is not a one-time measurement. It widens whenever your inputs change and you did not notice.

Re-measure when you enter a new market or language, when a new customer segment onboards with different data shapes, when you add a feature that accepts a new input type such as uploaded documents, when you upgrade the detector or its NER model, and on a schedule regardless, because gradual drift produces no event to trigger a check.

Make it a test that runs in CI. Load the golden set, run the detector, assert that per-entity recall has not dropped below the thresholds you committed to. A model upgrade that quietly halves recall on PERSON then fails a build instead of shipping.

[Fact]
public void PersonRecall_MeetsCommittedFloor()
{
    var scores = Evaluate(GoldenSet.ForEntity("PERSON"), Detector.Detect);
    Assert.True(scores.Recall >= 0.70,
        $"PERSON recall fell to {scores.Recall:P0}; committed floor is 70%.");
}

That test is also your evidence. When Chapter 17 asks what you can show an auditor about detector performance, this is the artefact, and it has been running every day since you wrote it.

You know what your detector finds, and you know what it misses. The next question is what to do with what it found, and the menu is considerably longer than replacing it with [REDACTED].

Sources for this chapter

• Per-entity accuracy spread (email 0.96–0.99 against person names 0.14–0.79 depending on corpus) — Sikkema benchmark, cited in full in Chapter 4: https://albertsikkema.com/python/security/privacy/2026/06/01/benchmarking-open-source-pii-detection.html
• ±5 character span-matching tolerance, and the finding that the top three models are not statistically separable — same source.

The threshold sweep table is illustrative and labelled as such in the text: the shape is real, the specific numbers are not measurements. The golden-set method, the 200–500 sample guidance, and the false-negative-versus-false-positive asymmetry are the author's practice rather than published findings. Treating the labelling set as a personal data processing operation follows from Chapter 2's definition, not from a cited source.