HF detector
davlan-xlmr PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
44.00%fully hidden annotations
51.33%detected / overlapped
0.71%extra masking
4,431CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
People's names91.16%59,923 / 65,736
Organizations69.71%13,435 / 19,274
Addresses & locations62.79%24,865 / 39,602
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Dates & times0.02%1 / 5,502
Passwords, keys & tokens0.03%2 / 6,870
Network identifiers0.05%14 / 25,631
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 44.86% | 51.97% | 0.66% |
| Russian | 19 | 40.44% | 47.64% | 0.84% |
| Multilingual | 5 | 52.49% | 60.27% | 0.42% |
Configuration, source and version
- Family
- HF
- Revision
- 253f557bd8249b8515114cfd7f71974fe5fa4d2f
- Eligible datasets
- 41 / 41
- Character F1
- 0.501
- Untouched diagnostic
- 48.67% · 110,716 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 2.257
- GPU seconds / 10k chars
- 0.136
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| arthur-passwords | 280 | 0.00% | 0.00% | 6.08% |
| corrupt-secrets-issues | 288 | 0.00% | 0.00% | 0.45% |
| creddata | 776 | 0.00% | 0.00% | 0.83% |
| leak-museum | 101 | 0.00% | 0.00% | 0.88% |
| leaky-repo | 95 | 0.00% | 0.00% | 0.42% |
| secrets-issues | 288 | 0.00% | 0.35% | 0.49% |
| secrets-rules | 746 | 0.00% | 0.00% | 0.09% |
| synth-env-configs | 392 | 0.00% | 0.00% | 0.00% |
| synth-secrets-en | 581 | 0.00% | 0.00% | 0.00% |
| synth-secrets-ru | 581 | 0.00% | 0.00% | 0.00% |
| rubai-ru | 3,458 | 4.45% | 55.21% | — |
| synth-ru-tickets | 32,099 | 10.24% | 16.15% | — |
| corrupt-hivetrace | 1,667 | 12.54% | 22.74% | 0.75% |
| hivetrace | 1,667 | 14.58% | 24.12% | 0.11% |
| synth-wiki-tables | 13,094 | 14.83% | 20.05% | — |
| synth-jira-comments | 2,705 | 21.00% | 21.77% | — |
| alexen2 | 1,261 | 22.36% | 45.60% | 11.01% |
| privy | 1,899 | 26.49% | 35.60% | 0.01% |
| multiconer-ru | 1,208 | 29.22% | 42.05% | 2.14% |
| nym-ru | 9,497 | 29.67% | 36.29% | — |
| nym-multi | 8,169 | 33.10% | 37.30% | 0.94% |
| corrupt-redmadrobot | 5,531 | 33.39% | 38.22% | 0.61% |
| ameau01 | 2,242 | 34.26% | 35.41% | 2.79% |
| russian-pii-66k | 4,805 | 35.46% | 38.98% | — |
| nym-en | 3,584 | 37.89% | 42.49% | 0.16% |
| redmadrobot | 5,516 | 39.63% | 42.60% | 0.47% |
| nemotron-pii | 9,391 | 44.55% | 49.61% | 0.00% |
| gretel-multi | 4,484 | 44.96% | 62.44% | 0.41% |
| redact-ru | 9,230 | 50.02% | 59.66% | 5.19% |
| tab-echr | 3,830 | 54.60% | 86.58% | — |
| redact-multi | 22,864 | 55.52% | 64.35% | — |
| tonicai | 2,417 | 55.73% | 60.16% | 0.17% |
| alrosait | 1,862 | 56.12% | 98.12% | 3.50% |
| kiji-multi | 7,565 | 57.62% | 58.94% | — |
| scanpatch | 8,708 | 58.21% | 62.23% | 1.11% |
| kiji-en | 7,623 | 60.04% | 62.44% | — |
| dialogpii-multi | 10,345 | 60.63% | 69.42% | — |
| dialogpii-en | 3,087 | 66.41% | 76.74% | 2.33% |
| jayguard | 1,195 | 67.20% | 82.93% | 1.24% |
| nerel | 24,369 | 84.10% | 88.84% | — |
| factrueval | 7,966 | 94.73% | 96.02% | — |