Home

/

Keep PII Out of Your LLM

/

The Leak You Can't See

The Leak You Can't See

Chapter 1
Part I
4
min read

The number that should bother you

Cyberhaven runs endpoint agents inside a lot of companies, which means it can watch what employees actually paste rather than what they say they paste. On its 2026 telemetry, 39.7% of enterprise AI interactions expose sensitive data. Not 39.7% of employees. Not 39.7% of companies. Four in ten of the individual interactions.

The direction matters more than the point estimate. The share of corporate data entering AI tools that qualifies as sensitive was 10.7% two years ago, 27.4% last year, and 34.8% now (Cyberhaven, 2026). It has roughly tripled while everyone was busy arguing about which model to use. Nearly 40% of the files people upload contain PII or payment card data, and 22% of pasted text contains information subject to some regulation.

The second number explains why your dashboards are quiet. Around 77% of employees paste data into generative AI prompts, and 82% of those pastes come from personal accounts outside company oversight. Your gateway logs, your enterprise tenant, your carefully negotiated data processing agreement: none of them see the four in five.

A caveat, because this book applies its own standards to itself. That is vendor telemetry from endpoint software, not a census and not a peer-reviewed survey. It is drawn from the population of companies who had already bought monitoring, which is not a neutral sample. Treat the exact figures as indicative. Treat the trend as real, because every independent source points the same way.

This is not a story about bad employees

The temptation is to read those numbers as a discipline problem and respond with a policy. Policies have been tried. They produce a training module, a signed acknowledgement, and no measurable change, because the behaviour is not deviant. It is the path of least resistance doing exactly what paths of least resistance do.

Think about what a person is actually doing when they paste a customer email thread into a chat window. They have a job to do, a tool that does it well, and a clipboard. The alternative is to spend four minutes manually stripping names from a message so they can spend thirty seconds getting a summary. Nobody does the four minutes. The friction is on the wrong side of the decision, so people route around it, and they route around it through an account you cannot see.

That is the consumer half of the problem, and it is mostly solved with procurement and sanctioned tooling. Give people a good enterprise option and much of the shadow traffic comes into the light.

The half this book is about is yours.

Your feature is one of the doors

Here is the part that catches engineering teams, and it has nothing to do with employees pasting anything.

You built a support-ticket summariser. The ticket object has a customer navigation property. You serialised the ticket, because serialising the ticket was one line and selecting fields would have been eleven. The prompt that leaves your service every few seconds now contains a full name, an email address, a postal address, a phone number, and whatever the customer typed into the free-text field, which in support tickets is frequently a date of birth, an account number, or a photograph of a document.

Nobody decided that. There is no meeting where someone said "let us send the postal address." It arrived because object graphs are convenient and prompts accept strings.

This pattern shows up everywhere once you look:

  • A CV screener that sends the whole document because parsing it into fields was going to be a sprint.
  • A "chat with your knowledge base" feature whose knowledge base is an export of Zendesk.
  • An agent that calls an internal API, gets back a customer record, and puts the record in its context for the next turn.
  • An error handler that attaches the request body to the exception so that debugging is easier.

Four different teams. Four different features. The same root cause: the sensitive value travelled because nothing in the code path was responsible for stopping it.

the-leak-you-cant-see
blast-radius
what-counts-as-pii
the-five-doors
the-accuracy-reckoning
the-hybrid-that-does-not-work
deterministic-detection
npi-in-c
the-three-way-choice
calling-the-analyzer-from-c
measuring-your-own-demo-gap
choosing-the-operating-point
the-ladder-of-safeguards
pseudonymisation
the-round-trip
restoring-safely
when-masking-breaks-the-task
plausibility-hazard
the-architecture-that-holds
the-reference-architecture
dont-send-it-at-all
structure-beats-prose
the-gateway
failure-is-a-policy-decision
the-sidecar-you-can-trust
egress-deny-it-at-the-network
rag-and-agents
de-identify-before-you-embed
dual-model-separation
the-boring-controls
evidence-and-the-first-thirty-days
week-two-the-chokepoint-and-the-fast-layer
entity-catalogue-and-c-validators
mbi-positional-rules
tooling-at-a-glance
azure-ai-language-pii-in-detail
container-trust-checklist
sources
azure-ai-language
provider-retention

Download the full PDF for free?

Free download — no account required

Get the PDF
Get the PDF
Related Chapters
Free Download
Get the full PDF
All pages, including all code examples, diagrams, and the appendix reference card.
No spam. Unsubscribe at any time.
Your email won't be shared.
Oops! There's a problem with your request. We're working on fixing it. Please try again later.