Consistent Pseudonymization for RAG Pipelines: Protect PII Without Breaking Retrieval

Why RAG Pipelines Break When You Naively Anonymize

Retrieval-Augmented Generation (RAG) depends on semantic similarity between a user's query and indexed document chunks. When a user asks "What did Maria Rossi order last month?", the retriever searches for chunks containing semantically relevant content — and the LLM synthesizes an answer from the retrieved context.

The problem arises at ingestion time. If you anonymize documents before embedding them, and your anonymization is not consistent, you break the semantic graph:

• Document A: "Maria Rossi" → [PERSON_1]
• Document B: "Maria Rossi" → [PERSON_3]
• Document C: "M. Rossi" → [PERSON_7]

Now the three documents that should co-retrieve together are indexed as referring to three different people. Your retrieval recall collapses. The LLM synthesizes incomplete answers from only a fraction of the relevant context.

The Problem: Inconsistent Anonymization Destroys Retrieval

Standard anonymization tools are designed for single-document, single-session use. They assign placeholder tokens sequentially ([PERSON_1], [PERSON_2], ...) without any persistence. Run the same document through the tool twice and you may get different token assignments.

For RAG pipelines, this means:

• Cross-document retrieval fails — the same entity has different tokens across documents, so similarity searches miss related chunks.
• Incremental indexing breaks — new documents added to the index use a fresh token sequence, creating collisions with existing tokens (a new [PERSON_1] may refer to a completely different person).
• Query-time anonymization mismatches — if you anonymize the user's query before retrieval, the query tokens must match the index tokens exactly.

The Solution: Consistent Pseudonymization

Consistent pseudonymization uses a deterministic mapping: the same input entity always produces the same output token. The mapping is stored in a persistent table (a database or key-value store) that is reused across all documents and sessions.

There are two implementation approaches:

Approach 1: Hash-Based Tokens

Compute a keyed hash of the entity value. The hash is always the same for the same input, so two occurrences of "Maria Rossi" produce identical tokens. Uses a secret key (HMAC) to prevent reverse-engineering.

import hmac, hashlib

HMAC_KEY = b"your-secret-key-32-bytes-minimum"

def consistent_token(entity_type: str, entity_value: str) -> str:
    """Produce a stable, deterministic token for any entity value."""
    digest = hmac.new(
        HMAC_KEY,
        f"{entity_type}:{entity_value.strip().lower()}".encode(),
        hashlib.sha256
    ).hexdigest()[:8]
    return f"[{entity_type}_{digest}]"

# Examples:
# consistent_token("PERSON", "Maria Rossi") -> "[PERSON_3a7f9c2b]"  (always)
# consistent_token("PERSON", "maria rossi") -> "[PERSON_3a7f9c2b]"  (same)
# consistent_token("PERSON", "John Smith")  -> "[PERSON_b1e4a8d6]"  (different)

Approach 2: Lookup Table with Sequential Assignment

Maintain a persistent dictionary mapping entity values to assigned tokens. On first encounter, assign the next available token for that type; on subsequent encounters, return the existing mapping.

import json, pathlib

class PersistentMapping:
    def __init__(self, path: str):
        self.path = pathlib.Path(path)
        self.data = json.loads(self.path.read_text()) if self.path.exists() else {}
        self.counters = {}

    def get_token(self, entity_type: str, entity_value: str) -> str:
        key = f"{entity_type}:{entity_value.strip().lower()}"
        if key not in self.data:
            count = self.counters.get(entity_type, 0) + 1
            self.counters[entity_type] = count
            self.data[key] = f"[{entity_type}_{count}]"
            self.path.write_text(json.dumps(self.data))
        return self.data[key]

    def reverse(self, token: str) -> str | None:
        reverse_map = {v: k.split(":", 1)[1] for k, v in self.data.items()}
        return reverse_map.get(token)

mapping = PersistentMapping("entity_mapping.json")
print(mapping.get_token("PERSON", "Maria Rossi"))  # [PERSON_1]
print(mapping.get_token("PERSON", "Maria Rossi"))  # [PERSON_1] (same)
print(mapping.get_token("PERSON", "John Smith"))   # [PERSON_2]

Using anonym.legal for Reversible Tokenization

anonym.legal offers reversible tokenization, for example through the anonym_legal_anonymize_text tool of its MCP server. Each call returns a session_id; the token mapping is stored, encrypted, for 24 hours by default or up to 30 days if you choose persistent. For corpus-wide consistency across many documents, keep your own mapping as shown above; for the REST interface, see the API documentation at anonym.legal.

// anonym.legal MCP server (hosted: https://anonym.legal/mcp)
// Tool: anonym_legal_anonymize_text
{
  "text": "Maria Rossi ordered from Berlin on Jan 15.",
  "mode": "tokenize",           // reversible replacement (default)
  "persistence": "persistent",  // mapping kept 30 days; "session" = 24 hours (default)
  "sessionName": "rag-corpus-v1"
}

// The response contains anonymized_text and a session_id.
// Keep the session_id: you need it to restore the original values.

Entity Types That Matter in RAG

Not all entity types affect retrieval equally. Focus consistent pseudonymization on entities that appear as query terms or cross-document references:

The Decrypt Workflow: Restoring Original Values After Retrieval

When the RAG pipeline returns an answer containing pseudonymized tokens, you need to restore the original values before showing the response to the user. This is the de-anonymization step:

// Tool: anonym_legal_detokenize_text
{
  "text": "<PERSON_1> placed order #<ID_1> on <DATE_1>.",
  "session_id": "<session_id from the anonymize_text response>"
}

// Returns the text with the original values restored,
// as long as the session has not expired.

The de-anonymization step should only run for authorized users. Implement role-based access control: analysts can query the anonymized index but only authorized data owners can see de-anonymized responses.

Compliance: GDPR Recital 26 and EU AI Act Article 10

GDPR Recital 26 states that anonymized data — data that cannot be attributed to an identified or identifiable person — falls outside the scope of GDPR. Consistent pseudonymization with a separately held, access-controlled mapping table does not meet this standard: under GDPR, pseudonymized data is still personal data (because re-identification is possible with the key), but it benefits from reduced obligations and is recognized as a security measure under Article 32.

EU AI Act Article 10 requires that training, validation and testing data for high-risk AI systems be subject to appropriate data governance practices, including measures to detect and correct biases and to ensure relevance and representativeness. Storing RAG corpora in consistently pseudonymized form — with a separate, access-controlled mapping table — can be one technical measure supporting that data governance and data protection by design (Article 25 GDPR). It does not by itself establish compliance with either provision.

Performance: Latency and Batch Processing

Calling a hosted service adds a network round trip to your ingestion pipeline:

• API latency: depends on text length, language and network; measure it in your own pipeline
• Batch mode: anonym.legal's batch API accepts 5, 20, 50 or 100 texts per request, depending on your plan; see the API documentation at anonym.legal
• Local caching: After the first call, cache the session mapping locally. Subsequent documents only hit the API for new entity values.
• Embedding overhead: Anonymized text has slightly different semantic properties than the original. Use the same embedding model throughout; do not mix embeddings of anonymized and non-anonymized text.

Limitations & Honest Caveats

Consistent pseudonymization has limitations in a RAG pipeline. Because the same entity always maps to the same token, an attacker who can correlate enough tokens across documents may still re-identify individuals — pseudonymized data remains personal data under GDPR. It is also not for every use case: workflows that need the real value at retrieval time (for example, contacting a named customer) require a separate, access-controlled de-anonymization step. Pseudonymization protects the index, not every downstream consumer of it.