RAG and Agents

The vector store is a database

Ask a team where their personal data lives and they will name the transactional database, probably the data warehouse, occasionally the object storage. Almost nobody names the vector index.

It is a database. It holds a copy of your support history, your document library, your case notes. It was populated by a pipeline nobody classified, it is backed up, it is replicated, and there is a good chance a copy exists in a staging environment because someone needed realistic data for testing retrieval quality.

The reason it escapes classification is that embeddings do not look like data. A vector is a list of floats. It appears to be a lossy, one-way transformation, and lossy one-way transformations feel like anonymisation.

They are not. Embedding inversion attacks recover 50% to 70% of the original input words from compromised vectors, and reconstruction approaches near-optimal accuracy with as few as a thousand samples against black-box encoders. OWASP lists vector and embedding weaknesses as a category in its own right: LLM08:2025, renumbered LLM09:2026 in the current edition.

Read that as the operational rule: treat your vector store as if it contains the plaintext, because for practical purposes it does. Encrypt it, control access to it, classify it, include it in your data inventory and your deletion processes, and do not copy it to staging.

Deletion, which is where this gets expensive

A person exercises their right to erasure. You delete their row. Where else are they?

In the vector index, as chunks of text they appear in, which may include documents authored by someone else that mention them. In any cached embedding. In a backup taken last Tuesday. In the fine-tuning set, if you built one, where they are now diffused into weights and cannot be removed at all.

The engineering answer is to design the index for deletion from the start:

• Store provenance on every chunk. Which source document, which subject, which tenant. A chunk with no lineage cannot be deleted on request.
• Prefer re-indexing a source to patching the index. Deleting the source and re-running ingestion is reliable; surgical vector deletion is not always supported and not always complete.
• Keep fine-tuning sets separate from retrieval corpora, and think very carefully before putting personal data in one, because that decision is irreversible.

De-identify before you embed

The strongest control for door two is the same as for door one: do not put it in.

Run your detection and de-identification pipeline over documents at ingestion time, not at query time. Ingestion is a batch job, so the latency argument disappears entirely. The 118 to 198 millisecond transformer detectors from Chapter 4 are perfectly affordable when you are processing a corpus overnight, and you can afford a lower threshold and a second pass.

public async Task IngestAsync(SourceDocument doc, CancellationToken ct)
{
    // Batch context: use the slower, higher-recall configuration.
    var findings = await _detector.AnalyzeAsync(
        doc.Text, policy: IngestionPolicy, ct: ct);

    var treated = Deidentify(doc.Text, findings);

    foreach (var chunk in Chunk(treated.Text))
    {
        await _index.UpsertAsync(new IndexEntry(
            Vector: await _embedder.EmbedAsync(chunk.Text, ct),
            Text: chunk.Text,
            SourceDocumentId: doc.Id,      // for deletion
            SubjectId: doc.SubjectId,      // for erasure requests
            Tenant: doc.Tenant), ct);      // for partitioning
    }
}

The three metadata fields at the bottom are not optional. They are how you answer an erasure request, how you scope retrieval, and how you prove which tenant's data is where.

Not everything can be de-identified before embedding, because sometimes the name is what makes the document retrievable. When that is true, the index holds personal data by design, and the controls move to access rather than content.