HF detector
nym-base PII detection benchmark
Measured on 38 of 41 retained datasets under the same frozen scoring and masking protocol.
64.53%fully hidden annotations
82.66%detected / overlapped
9.15%extra masking
3,116CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Logins & usernames97.35%10,531 / 10,818
Dates & times92.50%2,221 / 2,401
Phone numbers & email80.73%15,950 / 19,756
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers20.83%5,274 / 25,323
Other sensitive attributes30.59%865 / 2,828
Customer & employee IDs45.67%3,116 / 6,823
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 16 | 73.70% | 85.99% | 10.29% |
| Russian | 18 | 62.52% | 81.87% | 6.81% |
| Multilingual | 4 | 63.27% | 82.38% | 2.02% |
Configuration, source and version
- Family
- HF
- Revision
- 0ed0242d67a30ca6a83e5f118506ff1e1daf83ca
- Eligible datasets
- 38 / 41
- Character F1
- 0.672
- Untouched diagnostic
- 17.34% · 35,754 / 206,216
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 3.209
- GPU seconds / 10k chars
- 0.138
source overlap: nym-ru, nym-en, nym-multi
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| rubai-ru | 3,458 | 5.29% | 98.38% | — |
| alexen2 | 1,261 | 11.82% | 98.81% | 12.94% |
| alrosait | 1,862 | 18.21% | 99.36% | 4.38% |
| multiconer-ru | 1,208 | 32.45% | 46.69% | 5.71% |
| leak-museum | 101 | 34.65% | 46.53% | 3.29% |
| synth-jira-comments | 2,705 | 37.74% | 68.17% | — |
| synth-ru-tickets | 32,099 | 42.60% | 58.49% | — |
| tab-echr | 3,830 | 43.73% | 76.06% | — |
| ameau01 | 2,242 | 45.05% | 63.07% | 1.77% |
| gretel-multi | 4,484 | 45.63% | 82.25% | 2.02% |
| tonicai | 2,417 | 55.44% | 58.50% | 0.40% |
| dialogpii-multi | 10,345 | 55.49% | 72.36% | — |
| jayguard | 1,195 | 56.32% | 86.95% | 2.62% |
| synth-env-configs | 392 | 56.38% | 96.94% | 24.79% |
| dialogpii-en | 3,087 | 57.60% | 76.71% | 3.38% |
| redact-multi | 22,864 | 59.45% | 81.93% | — |
| redact-ru | 9,230 | 60.60% | 84.25% | 9.40% |
| synth-secrets-en | 581 | 62.48% | 93.98% | 32.00% |
| leaky-repo | 95 | 63.16% | 83.16% | 2.68% |
| corrupt-secrets-issues | 288 | 63.19% | 96.18% | 6.33% |
| synth-secrets-ru | 581 | 68.16% | 96.39% | 34.32% |
| synth-wiki-tables | 13,094 | 68.56% | 93.15% | — |
| secrets-issues | 288 | 69.44% | 98.61% | 6.72% |
| scanpatch | 8,708 | 70.69% | 91.09% | 1.45% |
| corrupt-redmadrobot | 5,531 | 71.27% | 83.02% | 1.92% |
| corrupt-hivetrace | 1,667 | 71.51% | 97.18% | 0.49% |
| hivetrace | 1,667 | 74.51% | 98.98% | 0.31% |
| nerel | 24,369 | 79.00% | 90.63% | — |
| redmadrobot | 5,516 | 80.53% | 89.12% | 2.10% |
| nemotron-pii | 9,391 | 82.99% | 90.76% | 5.22% |
| secrets-rules | 746 | 83.24% | 98.93% | 42.42% |
| privy | 1,899 | 83.36% | 96.37% | 11.50% |
| factrueval | 7,966 | 87.23% | 88.66% | — |
| creddata | 776 | 87.37% | 98.71% | 10.98% |
| kiji-en | 7,623 | 95.36% | 97.26% | — |
| kiji-multi | 7,565 | 95.92% | 97.51% | — |
| arthur-passwords | 280 | 97.86% | 97.86% | 16.16% |
| russian-pii-66k | 4,805 | 99.44% | 99.94% | — |