PII Detection Methods — Regex vs NLP vs Hybrid Approach

THREE APPROACHES

Method Overview

Every PII detection system uses one of three fundamental approaches — or a combination. Each has distinct strengths, limitations, and ideal use cases.

Regex / Pattern Matching

Deterministic rules that match known data formats. A card number either passes the Luhn check or it does not, so the same input always yields the same finding — no machine learning involved.

IBANs, credit cards, SSNs
Email addresses, phone numbers
IP addresses, passport numbers
Fast & deterministic

NLP / Named Entity Recognition

Machine learning models — spaCy (open-source NLP library), Stanza (Stanford NLP toolkit), and XLM-RoBERTa (cross-lingual transformer model) — that understand language context. Identifies entities that have no fixed format.

Person names, organizations
Locations, dates in context
Context-aware disambiguation
Handles unstructured text

Hybrid

Combines both engines plus context analysis and checksum validation, which adjust confidence scores. No accuracy or false-positive rate is published, because none has been benchmarked.

Merges NLP + Pattern results
Conflict resolution via confidence scoring
Checksum validation eliminates false positives
Best for production systems

UNDER THE HOOD

How Each Method Works

The difference between these approaches is not just accuracy — it is fundamental to what kinds of PII they can detect and how they handle ambiguity.

Regex Detection

Regex (regular expression) detection uses hand-crafted patterns to match known data formats character by character. It is entirely deterministic: the same input always produces the same output. Processing time is measured in microseconds, making it the fastest detection method available.

The limitation is structural: regex can only detect PII that follows a predictable format. It excels at IBANs, credit card numbers, Social Security numbers, email addresses, and phone numbers — but cannot detect a person's name, because names have no fixed pattern.

100% deterministic — identical results every run
No model downloads or ML infrastructure required
Limited to known, structured formats

EMAIL DETECTION PATTERN
/[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/IBAN DETECTION PATTERN
/[A-Z]{2}\d{2}[\s]?[\dA-Z]{4}[\s]?(?:[\dA-Z]{4}[\s]?){2,7}[\dA-Z]{1,4}/CREDIT CARD PATTERN (with Luhn check)
/\b(?:\d[\s-]*?){13,19}\b/+ Luhn checksum validation post-match

NLP Detection

NLP (Natural Language Processing) detection uses trained statistical models to recognize entity boundaries in context. Instead of matching patterns, the model reads surrounding words to determine whether a token is a person, organization, location, or date.

For example, in “Dr. Smith called John yesterday,” the model identifies “Smith” and “John” as PERSON entities based on context — not because names follow a pattern. This makes NLP essential for detecting PII that has no fixed format.

Context-aware — understands surrounding text
Handles ambiguity (“Jordan” as person vs. country)
Multi-language: spaCy, Stanza and transformer models across 48 languages (anonym.legal)
Requires model download and inference time

NLP ENGINE SELECTION

spaCy

PRIMARY — 25 languages, fastest inference

Stanza

SECONDARY — 7 languages not covered by spaCy

XLM-R

FALLBACK — 16 additional languages via transformer

Engine is auto-selected based on detected language. No overlap — each language belongs to exactly one NLP engine.

Hybrid Detection

Hybrid detection runs both the Regex and NLP engines in a single pass, then merges results using confidence scoring and context-aware recognition. When both engines detect the same span of text, the algorithm applies confidence scoring and conflict resolution to select the most accurate result.

Checksum validation on structured data (Luhn for credit cards, MOD-97 for IBANs) filters out false positives that pattern matching alone might produce. Context from the NLP engine resolves ambiguous detections. No accuracy or false-positive rate is published, because none has been benchmarked.

Runs both engines simultaneously
Confidence scoring merges overlapping detections
Checksum validation reduces structured data false positives
Confidence scoring enables precision/recall tuning

HYBRID MERGE PIPELINE

1

Run pattern recognizers (regex + checksums)

2

Run NLP Engine (auto-selected by language)

3

Merge results — resolve overlapping spans

4

Validate checksums (Luhn, MOD-97, format rules)

5

Apply confidence scoring & output final entities

COMPARISON

Method Comparison

Each detection method has different strengths. Regex excels at structured data, NLP excels at context-dependent entities, and Hybrid combines both for the highest coverage.

Characteristic	Regex Only	NLP Only	Hybrid
Structured Data	Checksum-verified	Not applicable	Checksum-verified
Name Detection	Very limited	Strong (context-aware)	Strong (context-aware)
Speed	Fastest	Medium	Medium
False Positives	Low (checksum validation)	Medium	Lowest
Language Support	Pattern-dependent	48 languages	48 languages
Setup Complexity	Simple	Model required	Managed
Context Awareness	None	High	Highest

Why NLP is essential for names

Regex cannot effectively detect names because names have no fixed format. A regex can match “Mr.” or “Dr.” prefixes, but cannot distinguish “Jordan” the person from “Jordan” the country, or identify “Smith” as a surname without surrounding context. This is precisely where NLP excels.

REAL-WORLD SCENARIOS

Where Each Method Wins

Different document types require different detection strategies. These real-world scenarios illustrate where hybrid detection finds what a single engine misses.

Medical Records

Medical documents contain a mix of patient names (NLP-dependent), medical record numbers (Regex-dependent), and diagnoses with contextual dates. Neither engine alone achieves full coverage.

                    Patient: [NAME] ← NLP

                    MRN: [MRN] ← Regex

                    Hybrid wins — catches both

Financial Documents

IBANs and credit card numbers are perfectly matched by Regex with checksum validation. But account holder names, beneficiary fields, and transaction descriptions require NLP to detect.

                    IBAN: [IBAN] ← Regex + MOD-97

                    Beneficiary: [NAME] ← NLP

                    Hybrid wins — validates + detects

Legal Contracts

Party names, physical addresses, dates of birth, and witness signatures are heavily context-dependent. Regex catches formatted tax IDs, but the majority of PII in contracts requires NLP.

                    Party: [ORG] ← NLP

                    Signatory: [NAME] ← NLP

                    NLP essential — context-heavy

Log Files

Server logs contain IP addresses, email addresses, and URLs that Regex handles perfectly. But user-agent strings and error messages may embed user names or file paths containing PII.

                    IP: [IP] ← Regex

                    User path: /home/[NAME]/ ← NLP

                    Hybrid catches both

OUR ARCHITECTURE

Two Products, Two Detection Engines

There is no single shared system: anonym.legal (hosted) and anonym.plus (local) each run their own engine, independently. Both combine pattern recognizers, NLP models and confidence scoring.

anonym.plus: Local Pattern Engine

317 own pattern recognizers, running on the customer’s machine; detection makes no network call. Each pattern includes format validation, and many include checksum verification (Luhn, MOD-97, format rules).

317 own pattern recognizers
Runs locally; detection makes no network call
Checksum validation built-in
Country-specific format rules

anonym.legal: Hosted NLP Engine

267 entity types across 48 languages, processed on our servers in Germany. The NLP model is chosen by language: spaCy for most languages, Stanza or transformer models for the rest.

267 entity types
48 languages
spaCy, Stanza and transformer models
Model selected by language

In Both: Confidence Scoring & Validation

The merge and validation layer. Resolves conflicts when both engines detect the same span, applies checksum validation to structured detections, and assigns confidence scores from 0.0 to 1.0.

Conflict resolution for overlapping spans
Checksum validation (Luhn, MOD-97)
Confidence scoring (0.0–1.0)
Language-specific error reduction

Deterministic, Not AI-Based

Our detection pipeline uses NLP models and regex patterns — not LLMs, and it does not depend on external AI APIs.

Production Impact

Checksum verification and context-aware recognition adjust confidence scores and filter out some false positives. We publish no accuracy or false-positive figure, and no comparison with baseline Microsoft Presidio (the open-source PII detection framework our engines build on), because none has been benchmarked.

Does this apply to your organisation?

The guide is general; your obligation is not. Tell us the sector and where the data is allowed to go, and we will tell you what applies.

Talk to us The two products

Try Detection Methods Live

See regex, NLP, and hybrid detection in action across real-world datasets:

Hosted — anonym.legal

Our servers in Germany. 267 entity types, 48 languages. Free tier, then from €3 a month.

Open anonym.legal ↗

On your machine — anonym.plus

Desktop client, the text never leaves the computer. 317 own pattern recognisers. Free for one machine, then a one-time licence from €149.

Open anonym.plus ↗

Compare both products →