HF detector
davlan-mbert PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
41.98%fully hidden annotations
48.88%detected / overlapped
0.94%extra masking
3,983CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
People's names88.69%58,303 / 65,736
Organizations66.78%12,871 / 19,274
Addresses & locations59.53%23,577 / 39,602
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Dates & times0.02%1 / 5,502
Network identifiers0.06%16 / 25,631
Passwords, keys & tokens0.12%8 / 6,870
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 41.99% | 49.32% | 0.95% |
| Russian | 19 | 38.91% | 45.37% | 0.94% |
| Multilingual | 5 | 49.83% | 57.55% | 0.30% |
Configuration, source and version
- Family
- HF
- Revision
- e756de7f7b8f64fea0c3d7c3872f1322fab747b1
- Eligible datasets
- 41 / 41
- Character F1
- 0.481
- Untouched diagnostic
- 51.12% · 116,270 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 2.511
- GPU seconds / 10k chars
- 0.147
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| corrupt-secrets-issues | 288 | 0.00% | 1.04% | 0.92% |
| creddata | 776 | 0.00% | 0.13% | 1.19% |
| leaky-repo | 95 | 0.00% | 0.00% | 0.67% |
| secrets-issues | 288 | 0.00% | 1.04% | 0.70% |
| synth-env-configs | 392 | 0.00% | 0.00% | 0.07% |
| synth-secrets-en | 581 | 0.00% | 0.00% | 0.01% |
| synth-secrets-ru | 581 | 0.00% | 0.00% | 0.05% |
| secrets-rules | 746 | 0.13% | 0.13% | 0.80% |
| arthur-passwords | 280 | 0.71% | 0.71% | 5.55% |
| leak-museum | 101 | 0.99% | 0.99% | 0.79% |
| rubai-ru | 3,458 | 4.05% | 55.12% | — |
| synth-ru-tickets | 32,099 | 8.24% | 11.70% | — |
| corrupt-hivetrace | 1,667 | 11.58% | 22.44% | 0.57% |
| synth-wiki-tables | 13,094 | 13.64% | 14.40% | — |
| hivetrace | 1,667 | 14.10% | 24.60% | 0.18% |
| synth-jira-comments | 2,705 | 20.00% | 20.78% | — |
| alexen2 | 1,261 | 20.86% | 44.89% | 10.05% |
| multiconer-ru | 1,208 | 24.17% | 37.00% | 2.01% |
| nym-ru | 9,497 | 28.46% | 35.91% | — |
| ameau01 | 2,242 | 29.17% | 33.76% | 3.48% |
| privy | 1,899 | 30.28% | 39.60% | 0.11% |
| nym-multi | 8,169 | 31.02% | 35.76% | 0.94% |
| russian-pii-66k | 4,805 | 31.26% | 36.27% | — |
| corrupt-redmadrobot | 5,531 | 32.60% | 36.18% | 0.73% |
| nym-en | 3,584 | 35.60% | 41.02% | 0.53% |
| redmadrobot | 5,516 | 38.23% | 41.15% | 0.53% |
| tonicai | 2,417 | 39.47% | 44.68% | 0.05% |
| gretel-multi | 4,484 | 42.28% | 60.66% | 0.29% |
| nemotron-pii | 9,391 | 42.95% | 47.69% | 0.00% |
| redact-ru | 9,230 | 49.02% | 58.19% | 6.54% |
| alrosait | 1,862 | 52.63% | 96.40% | 3.97% |
| redact-multi | 22,864 | 53.74% | 61.69% | — |
| tab-echr | 3,830 | 53.81% | 87.10% | — |
| dialogpii-multi | 10,345 | 54.99% | 64.60% | — |
| jayguard | 1,195 | 55.23% | 75.40% | 1.37% |
| kiji-multi | 7,565 | 55.72% | 57.05% | — |
| scanpatch | 8,708 | 55.81% | 60.51% | 1.12% |
| kiji-en | 7,623 | 56.33% | 57.96% | — |
| dialogpii-en | 3,087 | 62.97% | 72.92% | 2.33% |
| nerel | 24,369 | 83.56% | 88.76% | — |
| factrueval | 7,966 | 93.96% | 95.43% | — |