HF detector
mmbert32k PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
50.17%fully hidden annotations
78.53%detected / overlapped
10.33%extra masking
4,313CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Dates & times82.81%4,556 / 5,502
People's names74.29%48,838 / 65,736
Documents & identifiers61.41%9,232 / 15,034
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Organizations15.88%3,061 / 19,274
Other sensitive attributes20.30%651 / 3,207
Customer & employee IDs27.75%2,218 / 7,994
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 58.34% | 79.33% | 11.78% |
| Russian | 19 | 42.57% | 75.98% | 7.27% |
| Multilingual | 5 | 63.85% | 84.48% | 2.97% |
Configuration, source and version
- Family
- HF
- Revision
- d22c818cf9f2a8bfbbed8508cb417dc16a1ba3ea
- Eligible datasets
- 41 / 41
- Character F1
- 0.613
- Untouched diagnostic
- 21.47% · 48,833 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 2.319
- GPU seconds / 10k chars
- 0.183
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| rubai-ru | 3,458 | 5.67% | 98.38% | — |
| tab-echr | 3,830 | 14.54% | 47.39% | — |
| synth-secrets-en | 581 | 20.83% | 78.31% | 33.00% |
| synth-secrets-ru | 581 | 22.03% | 76.59% | 33.94% |
| alexen2 | 1,261 | 22.60% | 98.33% | 8.11% |
| leak-museum | 101 | 22.77% | 42.57% | 2.71% |
| synth-env-configs | 392 | 25.26% | 86.99% | 23.32% |
| multiconer-ru | 1,208 | 27.98% | 53.56% | 6.96% |
| synth-wiki-tables | 13,094 | 28.52% | 86.99% | — |
| nerel | 24,369 | 34.06% | 48.84% | — |
| leaky-repo | 95 | 34.74% | 66.32% | 3.15% |
| corrupt-secrets-issues | 288 | 35.07% | 83.33% | 8.84% |
| synth-ru-tickets | 32,099 | 39.99% | 80.56% | — |
| redact-ru | 9,230 | 40.94% | 78.21% | 9.26% |
| factrueval | 7,966 | 42.19% | 45.78% | — |
| alrosait | 1,862 | 43.98% | 98.23% | 3.27% |
| secrets-issues | 288 | 44.44% | 81.60% | 9.77% |
| dialogpii-en | 3,087 | 46.65% | 64.53% | 4.85% |
| synth-jira-comments | 2,705 | 47.28% | 88.54% | — |
| jayguard | 1,195 | 48.79% | 79.41% | 3.12% |
| dialogpii-multi | 10,345 | 49.71% | 68.42% | — |
| scanpatch | 8,708 | 50.42% | 83.77% | 1.34% |
| corrupt-redmadrobot | 5,531 | 50.99% | 79.30% | 2.83% |
| ameau01 | 2,242 | 52.05% | 77.79% | 4.59% |
| gretel-multi | 4,484 | 56.31% | 77.30% | 2.93% |
| secrets-rules | 746 | 56.43% | 93.83% | 48.11% |
| redmadrobot | 5,516 | 59.30% | 81.24% | 2.54% |
| privy | 1,899 | 60.40% | 77.62% | 6.08% |
| creddata | 776 | 60.70% | 89.43% | 10.19% |
| corrupt-hivetrace | 1,667 | 61.79% | 95.02% | 4.08% |
| nemotron-pii | 9,391 | 62.01% | 86.40% | 3.10% |
| tonicai | 2,417 | 62.10% | 73.85% | 0.40% |
| redact-multi | 22,864 | 62.39% | 89.25% | — |
| nym-ru | 9,497 | 63.43% | 91.51% | — |
| hivetrace | 1,667 | 74.75% | 96.70% | 0.37% |
| kiji-multi | 7,565 | 75.61% | 87.10% | — |
| russian-pii-66k | 4,805 | 75.96% | 97.36% | — |
| kiji-en | 7,623 | 77.45% | 87.16% | — |
| nym-en | 3,584 | 77.57% | 91.41% | 4.02% |
| nym-multi | 8,169 | 79.09% | 93.01% | 5.81% |
| arthur-passwords | 280 | 82.14% | 82.14% | 16.37% |