HF detector
kalyan-ettin PII detection benchmark
Measured on 40 of 41 retained datasets under the same frozen scoring and masking protocol.
43.35%fully hidden annotations
65.62%detected / overlapped
7.57%extra masking
8,742CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Logins & usernames85.28%9,511 / 11,153
Dates & times83.04%4,339 / 5,225
Passwords, keys & tokens57.98%3,766 / 6,495
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Organizations11.15%2,055 / 18,431
Other sensitive attributes14.62%469 / 3,207
Customer & employee IDs19.47%1,477 / 7,585
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 16 | 61.06% | 77.89% | 8.55% |
| Russian | 19 | 37.85% | 62.57% | 5.50% |
| Multilingual | 5 | 48.02% | 66.91% | 2.33% |
Configuration, source and version
- Family
- HF
- Revision
- 500262a2aaf913825ef750ef255c3fe437cd8e64
- Eligible datasets
- 40 / 41
- Character F1
- 0.579
- Untouched diagnostic
- 34.38% · 74,974 / 218,075
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 1.144
- GPU seconds / 10k chars
- Not measured
source overlap: nemotron-pii
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| rubai-ru | 3,458 | 2.69% | 65.12% | — |
| multiconer-ru | 1,208 | 9.60% | 26.66% | 5.11% |
| tab-echr | 3,830 | 12.27% | 52.27% | — |
| alexen2 | 1,261 | 14.20% | 91.59% | 12.43% |
| alrosait | 1,862 | 19.92% | 74.81% | 4.55% |
| jayguard | 1,195 | 29.21% | 52.47% | 2.13% |
| nerel | 24,369 | 29.56% | 50.56% | — |
| corrupt-redmadrobot | 5,531 | 30.34% | 43.64% | 1.65% |
| dialogpii-multi | 10,345 | 32.18% | 44.74% | — |
| synth-wiki-tables | 13,094 | 33.47% | 75.51% | — |
| corrupt-hivetrace | 1,667 | 37.67% | 66.41% | 2.12% |
| leak-museum | 101 | 38.61% | 46.53% | 3.81% |
| synth-ru-tickets | 32,099 | 39.28% | 64.72% | — |
| factrueval | 7,966 | 40.80% | 44.11% | — |
| scanpatch | 8,708 | 42.55% | 66.32% | 1.79% |
| gretel-multi | 4,484 | 43.09% | 79.71% | 2.32% |
| redmadrobot | 5,516 | 44.18% | 56.74% | 1.24% |
| redact-multi | 22,864 | 44.22% | 68.89% | — |
| synth-secrets-ru | 581 | 44.23% | 85.20% | 26.93% |
| redact-ru | 9,230 | 44.88% | 71.64% | 7.87% |
| ameau01 | 2,242 | 45.67% | 68.29% | 2.63% |
| synth-env-configs | 392 | 46.68% | 92.35% | 15.88% |
| synth-secrets-en | 581 | 48.02% | 85.54% | 25.69% |
| leaky-repo | 95 | 49.47% | 76.84% | 2.89% |
| synth-jira-comments | 2,705 | 49.54% | 76.56% | — |
| dialogpii-en | 3,087 | 50.31% | 68.71% | 5.45% |
| hivetrace | 1,667 | 51.41% | 72.77% | 1.06% |
| nym-ru | 9,497 | 53.39% | 69.58% | — |
| corrupt-secrets-issues | 288 | 53.47% | 86.81% | 6.49% |
| privy | 1,899 | 55.45% | 74.67% | 4.79% |
| secrets-rules | 746 | 57.51% | 85.39% | 39.29% |
| nym-multi | 8,169 | 59.48% | 70.36% | 2.62% |
| secrets-issues | 288 | 60.42% | 88.19% | 6.59% |
| russian-pii-66k | 4,805 | 61.79% | 76.84% | — |
| tonicai | 2,417 | 69.01% | 72.78% | 1.28% |
| kiji-multi | 7,565 | 71.75% | 79.96% | — |
| nym-en | 3,584 | 76.93% | 87.44% | 2.02% |
| creddata | 776 | 77.84% | 92.01% | 4.71% |
| kiji-en | 7,623 | 86.00% | 91.07% | — |
| arthur-passwords | 280 | 88.21% | 88.21% | 16.10% |