When Masking Breaks the Task

The failure is silent

A model given [REDACTED] where a date should be does not say "I cannot answer without the date." It writes a summary that omits the date, or invents a plausible one, or reasons about a value it does not have. The output looks exactly like a good output.

That is what makes this failure mode dangerous compared to the ones in Chapter 9. A mangled restore produces visible gibberish. A broken task produces a clean, fluent, wrong answer, and nobody investigates a clean answer.

Four patterns account for most of it.

Numeric ambiguity

You redact a number. The model no longer knows what kind of number it was, and neither does your detector.

Consider 47 in a clinical note. It could be an age, a dose in milligrams, a bed number, a systolic reading, or the patient's weight. A detector tuned to catch ages will catch all of them, because at the character level they are identical. Redact them all and the note is useless. Redact none and you leak the age.

This is a genuinely hard problem and there is research on it specifically, in tutoring and clinical contexts, precisely because numbers carry both the identifying information and the task-critical information in the same token.

Two practical moves. Use context, not the value. A number preceded by "aged" or followed by "years old" is an age; a number inside a dose: field is not. Presidio's contextual scoring does some of this and you can extend it with your own recognisers, which is one more argument for a framework you can plug into. Generalise rather than redact. Safeguard 4 from Chapter 8 keeps the number's category and blurs its precision. 47 becomes 40-49, the note stays readable, the model can still reason about an adult in middle age, and you have removed a strong quasi-identifier.

Coreference collapse

Covered in Chapter 8 and worth the repetition because it is the most common cause of degraded output.

Redaction maps many people onto one marker. An email thread with four participants becomes a thread about [PERSON] talking to [PERSON] about [PERSON]. Any summary of it is fiction.

Pseudonymisation fixes this and is why it is the default. <PERSON_1> through <PERSON_4> keeps the relationships intact. If you are using redaction on anything conversational, this is almost certainly why your summaries are poor, and the fix is a safeguard on the ladder rather than a better prompt.

Signal in the name

Sometimes the identifier is the thing the task is about.

A model asked to detect potential bias in a hiring process needs to see the names, because the names are the variable under examination. A model asked to check whether a translation preserved a person's name needs the name. A model routing correspondence by language may be using the name as a signal.

Notice that the first example is also the classic argument for masking: Microsoft's own documentation lists masking identifiers in resume screening to reduce bias as a use case, and it is a good one. Both are true. Blind the screening decision, and show the names to the audit that checks whether the screening was biased. Same data, opposite safeguards, because the tasks are different.

This is the general principle: decide per task, never per system. A single global redaction policy will be wrong for some task in your application, and it will be wrong silently.