HF detector
openmed-multilingual PII detection benchmark
Measured on 38 of 41 retained datasets under the same frozen scoring and masking protocol.
51.75%fully hidden annotations
72.55%detected / overlapped
11.92%extra masking
3,076CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Dates & times84.92%4,420 / 5,205
Passwords, keys & tokens77.42%4,837 / 6,248
Phone numbers & email76.60%15,598 / 20,363
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Organizations10.68%1,911 / 17,900
Other sensitive attributes28.44%912 / 3,207
Addresses & locations43.52%16,020 / 36,811
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 15 | 69.76% | 84.03% | 13.49% |
| Russian | 19 | 45.59% | 69.32% | 8.51% |
| Multilingual | 4 | 59.23% | 75.34% | 6.93% |
Configuration, source and version
- Family
- HF
- Revision
- 0d0c0430fa386435ace1d9f842f0f966903a9ac8
- Eligible datasets
- 38 / 41
- Character F1
- 0.626
- Untouched diagnostic
- 27.45% · 58,119 / 211,692
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 3.251
- GPU seconds / 10k chars
- 0.837
source overlap: nemotron-pii, gretel-multi, privy
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| rubai-ru | 3,458 | 3.96% | 92.74% | — |
| alexen2 | 1,261 | 7.06% | 88.50% | 9.27% |
| multiconer-ru | 1,208 | 7.28% | 22.10% | 6.82% |
| alrosait | 1,862 | 8.75% | 80.50% | 5.36% |
| nerel | 24,369 | 18.93% | 35.77% | — |
| tab-echr | 3,830 | 24.15% | 64.26% | — |
| jayguard | 1,195 | 28.70% | 58.74% | 2.90% |
| factrueval | 7,966 | 31.77% | 34.31% | — |
| corrupt-redmadrobot | 5,531 | 36.14% | 52.68% | 3.67% |
| dialogpii-multi | 10,345 | 40.57% | 56.92% | — |
| redmadrobot | 5,516 | 47.86% | 61.93% | 2.84% |
| scanpatch | 8,708 | 51.40% | 76.22% | 2.34% |
| synth-jira-comments | 2,705 | 51.87% | 66.84% | — |
| ameau01 | 2,242 | 52.59% | 71.72% | 4.37% |
| synth-ru-tickets | 32,099 | 53.21% | 84.69% | — |
| redact-ru | 9,230 | 53.24% | 77.94% | 9.08% |
| corrupt-hivetrace | 1,667 | 53.75% | 84.04% | 3.69% |
| corrupt-secrets-issues | 288 | 53.82% | 89.58% | 9.59% |
| redact-multi | 22,864 | 54.48% | 77.67% | — |
| leak-museum | 101 | 55.45% | 62.38% | 5.07% |
| dialogpii-en | 3,087 | 56.24% | 78.62% | 6.43% |
| leaky-repo | 95 | 58.95% | 90.53% | 6.82% |
| synth-wiki-tables | 13,094 | 61.81% | 84.60% | — |
| tonicai | 2,417 | 63.43% | 68.47% | 1.37% |
| synth-env-configs | 392 | 64.29% | 98.98% | 22.80% |
| secrets-issues | 288 | 65.97% | 93.06% | 10.31% |
| hivetrace | 1,667 | 69.77% | 92.20% | 1.82% |
| synth-secrets-ru | 581 | 72.12% | 96.56% | 39.35% |
| secrets-rules | 746 | 74.13% | 87.94% | 47.26% |
| creddata | 776 | 75.77% | 91.24% | 10.54% |
| nym-ru | 9,497 | 76.77% | 88.13% | — |
| nym-multi | 8,169 | 77.39% | 84.78% | 6.93% |
| kiji-multi | 7,565 | 79.50% | 83.30% | — |
| russian-pii-66k | 4,805 | 80.17% | 88.05% | — |
| synth-secrets-en | 581 | 82.44% | 99.14% | 41.95% |
| nym-en | 3,584 | 91.29% | 95.90% | 5.25% |
| kiji-en | 7,623 | 93.31% | 95.17% | — |
| arthur-passwords | 280 | 100.00% | 100.00% | 24.78% |