HF detector
mmbert32k+cpu-int8 PII detection benchmark
Measured on 9 of 41 retained datasets under the same frozen scoring and masking protocol.
21.91%fully hidden annotations
63.70%detected / overlapped
6.87%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Bank accounts & cards62.02%1,166 / 1,880
Documents & identifiers52.60%3,120 / 5,931
Passwords, keys & tokens37.95%639 / 1,684
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers1.47%277 / 18,902
Organizations5.63%257 / 4,568
Other sensitive attributes10.27%23 / 224
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 4 | 25.70% | 58.42% | 8.55% |
| Russian | 4 | 16.19% | 64.84% | 2.22% |
| Multilingual | 1 | 50.89% | 67.36% | — |
Configuration, source and version
- Family
- HF
- Revision
- d22c818cf9f2a8bfbbed8508cb417dc16a1ba3ea
- Eligible datasets
- 9 / 41
- Character F1
- 0.416
- Untouched diagnostic
- 36.30% · 26,169 / 72,095
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- Not measured
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| tab-echr | 3,830 | 5.93% | 35.22% | — |
| synth-ru-tickets | 32,099 | 7.97% | 64.69% | — |
| ameau01 | 2,242 | 19.45% | 53.93% | 3.41% |
| nym-ru | 9,497 | 27.20% | 63.81% | — |
| nemotron-pii | 9,391 | 34.33% | 67.85% | 2.25% |
| redmadrobot | 5,516 | 34.45% | 60.88% | 2.50% |
| kiji-multi | 7,565 | 50.89% | 67.36% | — |
| hivetrace | 1,667 | 51.29% | 86.68% | 0.37% |
| secrets-issues | 288 | 55.90% | 94.44% | 9.56% |