HF detector
mmbert32k+nochunk PII detection benchmark
Measured on 11 of 41 retained datasets under the same frozen scoring and masking protocol.
21.11%fully hidden annotations
51.52%detected / overlapped
6.78%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Dates & times85.36%513 / 601
Bank accounts & cards68.02%519 / 763
Documents & identifiers51.77%1,011 / 1,953
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers1.95%341 / 17,498
Organizations7.61%999 / 13,123
Other sensitive attributes11.59%103 / 889
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 6 | 40.68% | 65.86% | 7.34% |
| Russian | 5 | 16.54% | 48.17% | 3.10% |
Configuration, source and version
- Family
- HF
- Revision
- d22c818cf9f2a8bfbbed8508cb417dc16a1ba3ea
- Eligible datasets
- 11 / 41
- Character F1
- 0.412
- Untouched diagnostic
- 48.48% · 44,764 / 92,332
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- 0.250
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| tab-echr | 3,830 | 8.72% | 38.17% | — |
| synth-ru-tickets | 32,099 | 10.16% | 58.29% | — |
| nerel | 24,369 | 17.74% | 34.40% | — |
| leak-museum | 101 | 21.78% | 41.58% | 2.51% |
| factrueval | 7,966 | 24.50% | 27.13% | — |
| redact-ru | 9,230 | 24.88% | 63.89% | 6.47% |
| dialogpii-en | 3,087 | 26.50% | 41.53% | 3.53% |
| jayguard | 1,195 | 45.86% | 75.90% | 3.01% |
| secrets-issues | 288 | 46.53% | 83.68% | 7.24% |
| nemotron-pii | 9,391 | 56.77% | 82.95% | 3.10% |
| creddata | 776 | 60.44% | 89.05% | 8.89% |