HF detector
fef2-secret-ru PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
35.60%fully hidden annotations
43.35%detected / overlapped
2.17%extra masking
3,278CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Logins & usernames76.69%8,729 / 11,382
Passwords, keys & tokens53.67%3,687 / 6,870
People's names48.92%32,157 / 65,736
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers0.14%35 / 25,631
Dates & times1.22%67 / 5,502
Other sensitive attributes3.43%110 / 3,207
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 14.31% | 17.68% | 2.35% |
| Russian | 19 | 52.11% | 63.18% | 1.84% |
| Multilingual | 5 | 8.42% | 10.80% | 0.07% |
Configuration, source and version
- Family
- HF
- Revision
- 52b5b0745aac14f73fcf2ac0f91d9b5001a85ae4
- Eligible datasets
- 41 / 41
- Character F1
- 0.530
- Untouched diagnostic
- 56.65% · 128,858 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 3.051
- GPU seconds / 10k chars
- 0.149
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| dialogpii-multi | 10,345 | 0.75% | 0.99% | — |
| dialogpii-en | 3,087 | 1.17% | 1.78% | 0.00% |
| tab-echr | 3,830 | 2.06% | 9.09% | — |
| kiji-multi | 7,565 | 4.68% | 5.68% | — |
| rubai-ru | 3,458 | 5.81% | 73.11% | — |
| privy | 1,899 | 6.42% | 7.53% | 0.75% |
| gretel-multi | 4,484 | 9.63% | 12.93% | 0.07% |
| tonicai | 2,417 | 10.92% | 14.23% | 0.06% |
| ameau01 | 2,242 | 10.97% | 12.76% | 1.21% |
| nym-en | 3,584 | 11.10% | 12.14% | 0.71% |
| redact-multi | 22,864 | 11.71% | 15.60% | — |
| nym-multi | 8,169 | 11.74% | 13.36% | 0.00% |
| kiji-en | 7,623 | 13.71% | 15.18% | — |
| nemotron-pii | 9,391 | 14.94% | 17.72% | 0.00% |
| alexen2 | 1,261 | 22.20% | 73.43% | 8.82% |
| multiconer-ru | 1,208 | 24.42% | 36.26% | 2.68% |
| synth-wiki-tables | 13,094 | 24.84% | 39.22% | — |
| synth-jira-comments | 2,705 | 30.13% | 33.42% | — |
| secrets-issues | 288 | 32.99% | 51.39% | 0.44% |
| corrupt-secrets-issues | 288 | 33.33% | 51.04% | 0.43% |
| leaky-repo | 95 | 34.74% | 42.11% | 0.02% |
| synth-ru-tickets | 32,099 | 34.92% | 42.90% | — |
| secrets-rules | 746 | 43.03% | 50.40% | 28.08% |
| corrupt-redmadrobot | 5,531 | 44.59% | 51.37% | 0.70% |
| corrupt-hivetrace | 1,667 | 45.53% | 59.87% | 0.89% |
| nym-ru | 9,497 | 48.04% | 58.16% | — |
| leak-museum | 101 | 51.49% | 59.41% | 0.37% |
| synth-secrets-en | 581 | 53.18% | 71.08% | 4.40% |
| redact-ru | 9,230 | 54.28% | 68.21% | 7.27% |
| jayguard | 1,195 | 54.48% | 79.67% | 1.22% |
| redmadrobot | 5,516 | 55.00% | 60.71% | 1.12% |
| alrosait | 1,862 | 55.37% | 96.56% | 4.27% |
| synth-secrets-ru | 581 | 56.11% | 74.18% | 5.22% |
| hivetrace | 1,667 | 56.39% | 67.79% | 0.22% |
| synth-env-configs | 392 | 57.40% | 91.07% | 14.09% |
| scanpatch | 8,708 | 59.46% | 67.26% | 1.14% |
| arthur-passwords | 280 | 60.00% | 60.00% | 3.78% |
| creddata | 776 | 63.40% | 65.59% | 2.63% |
| russian-pii-66k | 4,805 | 67.35% | 81.42% | — |
| nerel | 24,369 | 83.10% | 89.08% | — |
| factrueval | 7,966 | 95.33% | 96.59% | — |