HF detector
ru-legal-ner PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
44.15%fully hidden annotations
65.45%detected / overlapped
12.56%extra masking
50,942CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Phone numbers & email69.13%15,196 / 21,981
Documents & identifiers60.82%9,143 / 15,034
People's names60.49%39,765 / 65,736
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers3.09%793 / 25,631
Customer & employee IDs22.82%1,824 / 7,994
Other sensitive attributes28.13%902 / 3,207
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 25.97% | 41.84% | 12.28% |
| Russian | 19 | 49.63% | 73.87% | 13.03% |
| Multilingual | 5 | 42.96% | 60.59% | 16.57% |
Configuration, source and version
- Family
- HF
- Revision
- c0313a42ac147ccc38f5c3b0b3e77cab53208683
- Eligible datasets
- 41 / 41
- Character F1
- 0.449
- Untouched diagnostic
- 34.55% · 78,588 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 0.196
- GPU seconds / 10k chars
- 0.131
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| tab-echr | 3,830 | 5.93% | 19.58% | — |
| dialogpii-en | 3,087 | 6.51% | 14.74% | 1.47% |
| rubai-ru | 3,458 | 6.74% | 89.97% | — |
| ameau01 | 2,242 | 16.37% | 30.78% | 1.87% |
| alexen2 | 1,261 | 23.79% | 94.13% | 6.83% |
| tonicai | 2,417 | 25.16% | 31.86% | 3.74% |
| nemotron-pii | 9,391 | 26.50% | 47.12% | 21.69% |
| leak-museum | 101 | 26.73% | 49.50% | 5.36% |
| synth-ru-tickets | 32,099 | 27.13% | 69.68% | — |
| multiconer-ru | 1,208 | 27.40% | 48.92% | 10.94% |
| gretel-multi | 4,484 | 27.99% | 56.22% | 16.63% |
| kiji-en | 7,623 | 28.02% | 36.76% | — |
| synth-env-configs | 392 | 28.57% | 86.22% | 21.58% |
| leaky-repo | 95 | 32.63% | 70.53% | 9.14% |
| nym-en | 3,584 | 33.43% | 49.80% | 6.25% |
| synth-secrets-en | 581 | 33.56% | 89.16% | 27.21% |
| privy | 1,899 | 35.18% | 51.50% | 11.64% |
| synth-secrets-ru | 581 | 35.63% | 91.05% | 29.02% |
| redact-multi | 22,864 | 39.29% | 61.07% | — |
| corrupt-secrets-issues | 288 | 41.67% | 77.43% | 11.76% |
| kiji-multi | 7,565 | 43.32% | 53.97% | — |
| synth-jira-comments | 2,705 | 43.44% | 77.93% | — |
| dialogpii-multi | 10,345 | 45.39% | 55.80% | — |
| nerel | 24,369 | 50.33% | 57.17% | — |
| factrueval | 7,966 | 53.41% | 56.18% | — |
| secrets-issues | 288 | 53.47% | 84.38% | 9.27% |
| nym-multi | 8,169 | 58.01% | 73.88% | 12.73% |
| corrupt-redmadrobot | 5,531 | 58.02% | 74.29% | 17.51% |
| redact-ru | 9,230 | 58.32% | 77.36% | 18.34% |
| jayguard | 1,195 | 59.00% | 77.82% | 3.99% |
| redmadrobot | 5,516 | 59.86% | 75.36% | 8.61% |
| scanpatch | 8,708 | 62.21% | 80.20% | 1.70% |
| synth-wiki-tables | 13,094 | 63.46% | 94.07% | — |
| nym-ru | 9,497 | 63.66% | 79.70% | — |
| secrets-rules | 746 | 65.01% | 96.92% | 36.25% |
| creddata | 776 | 68.17% | 89.95% | 13.31% |
| corrupt-hivetrace | 1,667 | 68.87% | 88.54% | 22.36% |
| hivetrace | 1,667 | 71.75% | 88.42% | 0.68% |
| alrosait | 1,862 | 72.45% | 96.29% | 3.86% |
| arthur-passwords | 280 | 80.00% | 80.00% | 10.36% |
| russian-pii-66k | 4,805 | 86.72% | 94.03% | — |