AI & privacy

RAG PII redaction before data enters the vector database

RAG PII redaction has to happen before indexing: whatever is stored as an embedding and text chunk will later surface in answers. Diskretio cleans source documents in the ingest step.

anonymize.shresponse.json

# Anonymize a file, report as JSON

$ curl -X POST "$API/v1/files?response=json" \

-H "x-api-key: $ANON_KEY" \

-F "file=@Employment_Contract_Smith.pdf" \

-F 'options={"strategy":"placeholder","language":"en"}'

HTTP/1.1 200 OK · x-pages-charged: 3 · x-pages-remaining: 1757

{

"filename": "anonymized.pdf",

"pages": 3,

"report": {

"counts": {

"PERSON": 6,

"ORG": 2,

"ADDRESS": 3,

"IBAN": 1,

"TAX_ID": 1,

"NATIONAL_ID": 1,

"EMAIL": 1

},

"entities": [{ "id": "PERSON_1", "label": "PERSON", "occurrences": 4 }, …],

"warnings": []

},

"content_base64": "JVBERi0xLjcKJeLjz9MKMSAwIG9iago8PC9UeXBl…"

}

# page 1 of the output

Between [ORG_1] and [PERSON_1], [ADDRESS_1] …

Your formats and the AI tools you work with afterwards

  • Word
  • Excel
  • PowerPoint
  • Outlook (emails)
  • PDF
  • Scans & photos
  • TXT Text & CSV
  • ChatGPT
  • Claude
  • Microsoft Copilot
  • Gemini
  • Mistral
  • DeepL
  • Perplexity

Where personal data sits in a RAG pipeline

  • Source documents: contracts, manuals with contact persons, minutes
  • Text chunks stored alongside the vectors
  • Metadata fields such as author, creator, file paths containing names
  • Scanned PDFs whose text only appears via OCR
  • Tables and CSV exports with customer lists
  • Comments and tracked changes in Office files

How it works

  1. 1

    Anonymize at ingest

    Send each document to the Diskretio API before chunking. Files come back in their original format or as JSON.

  2. 2

    Index the clean version

    Chunking and embedding run on cleaned text. Placeholders like [ORG_1] stay meaningful for retrieval.

  3. 3

    Answers without real names

    The language model only receives clean chunks and cannot repeat names it has never seen.

Why filtering afterwards is not enough

Some pipelines only filter personal data in the model’s answer. By then it already sits in the vector database, in backups and possibly at the embedding provider. GDPR erasure requests then have to be honored in every copy.

Anonymizing at ingest keeps the entire index free of personal data. That simplifies permissions, deletion concepts and the choice of embedding and model provider.

What placeholders cost you

Searching for a specific customer name no longer works in an anonymized index, and placeholders like [PERSON_1] are assigned per document, not across documents. For knowledge bases about processes, rules and case patterns that is usually what you want; for a customer CRM with name search, RAG on anonymized data is the wrong tool.

Terms that should stay, such as product names or authorities, go on an allow list. Public bodies are kept by default.

FAQ

Questions. Answered briefly.

Try it with your own document

10 pages per month are free. No subscription, no credit card.