Home

/

Keep PII Out of Your LLM

/

Evidence, and the First Thirty Days

Evidence, and the First Thirty Days

Chapter 17
Part V
4
min read
The thirty-day sequence across four weeks: measure and minimise, chokepoint and fast layer, sidecar and round trip, then evidence and re-measurement

What people actually ask for

Three audiences will ask, and they want the same five things in different tones of voice.

An enterprise customer's security questionnaire wants to know whether their data reaches a third-party model and under what controls. A regulator or DPO wants a lawful basis, a minimisation argument, and a record of processing. An auditor wants to see that the control you described was operating on a given date.

The five artefacts:

1. A data flow record. For each LLM feature: what personal data enters the prompt, what enters any corpus, which provider receives it, under what retention terms, in which region. This is Chapter 2's inventory, kept current.

2. The policy, versioned. Which entities are detected, at what threshold, treated how, on which routes, with a version string and a change history. Chapter 13 produced this as configuration in source control, which means the history is git log.

3. Detector performance on your own data. Precision and recall per entity type, measured on a golden set drawn from your traffic, with the measurement date and the sample size. Chapter 7 built this and put it in CI, so it is current by construction.

4. Proof the control was running. The gateway audit record: request, tenant, route, policy version, finding counts, whether it degraded. Not the content. This is what answers "was the control in force on 3 March?", and it answers it precisely.

5. A minimisation argument. Why each field in the prompt is necessary. Chapter 12 put this in a comment on the projection record, where it stays maintained.

Notice what is not on the list: a policy document nobody reads, a spreadsheet of controls maintained by hand, a slide deck. Every one of the five is generated by the system doing its job. That is the difference between evidence you have and evidence you assemble in a panic.

The two formal documents

A DPIA is required under GDPR where processing is likely to result in high risk, and an LLM feature processing personal data at scale frequently qualifies. Your DPO decides. What engineering owes it is the substance: the data flow record, the minimisation argument, the residual risk after controls, and the controls themselves. Artefacts 1, 2, 3 and 5 are most of a DPIA already.

Article 50 transparency, applicable since 2 August 2026, requires that people are told when they are interacting with an AI system. If you have a user-facing chatbot, this is a line of interface copy and a decision about where it appears. It is the most easily satisfied obligation in this book and the one most likely to be forgotten, because it is a product task rather than an engineering one.

Week one: measure and minimise

Do not start with the detector. Start with what you are sending.

Inventory every LLM call. Find them all, including the one in the batch job and the one somebody added to the admin tool. For each, capture a real prompt and read it. Teams routinely find a field they did not know they were sending, and the discovery is free to fix.

Write the projections. Chapter 12. For each feature, define the record that contains only what the task needs, with a comment per field saying why. This is the largest risk reduction available to you and it is a day or two of work.

Turn off prompt and completion logging, and add the assertion test. Chapter 16. An afternoon, and it closes the widest door.

Ask procurement to confirm retention, training-use and region with your provider. It runs in the background for the rest of the month.

By Friday you have measurably reduced exposure and you have not deployed a single new component.

the-leak-you-cant-see
blast-radius
what-counts-as-pii
the-five-doors
the-accuracy-reckoning
the-hybrid-that-does-not-work
deterministic-detection
npi-in-c
the-three-way-choice
calling-the-analyzer-from-c
measuring-your-own-demo-gap
choosing-the-operating-point
the-ladder-of-safeguards
pseudonymisation
the-round-trip
restoring-safely
when-masking-breaks-the-task
plausibility-hazard
the-architecture-that-holds
the-reference-architecture
dont-send-it-at-all
structure-beats-prose
the-gateway
failure-is-a-policy-decision
the-sidecar-you-can-trust
egress-deny-it-at-the-network
rag-and-agents
de-identify-before-you-embed
dual-model-separation
the-boring-controls
evidence-and-the-first-thirty-days
week-two-the-chokepoint-and-the-fast-layer
entity-catalogue-and-c-validators
mbi-positional-rules
tooling-at-a-glance
azure-ai-language-pii-in-detail
container-trust-checklist
sources
azure-ai-language
provider-retention

Download the full PDF for free?

Free download — no account required

Get the PDF
Get the PDF
Related Chapters
Free Download
Get the full PDF
All pages, including all code examples, diagrams, and the appendix reference card.
No spam. Unsubscribe at any time.
Your email won't be shared.
Oops! There's a problem with your request. We're working on fixing it. Please try again later.