HF detector
openmed-nemotron PII detection benchmark
Measured on 40 of 41 retained datasets under the same frozen scoring and masking protocol.
48.55%fully hidden annotations
70.61%detected / overlapped
12.23%extra masking
3,287CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Dates & times86.18%4,503 / 5,225
Logins & usernames79.25%8,839 / 11,153
Phone numbers & email75.81%15,755 / 20,782
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Organizations13.98%2,576 / 18,431
Other sensitive attributes19.46%624 / 3,207
Customer & employee IDs22.00%1,669 / 7,585
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 16 | 61.48% | 76.73% | 13.61% |
| Russian | 19 | 45.68% | 70.14% | 9.43% |
| Multilingual | 5 | 49.04% | 68.56% | 2.69% |
Configuration, source and version
- Family
- HF
- Revision
- f6f3a633dc6fc96a644d20218633a98ac43f6c71
- Eligible datasets
- 40 / 41
- Character F1
- 0.574
- Untouched diagnostic
- 29.39% · 64,103 / 218,075
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 3.042
- GPU seconds / 10k chars
- 0.874
source overlap: nemotron-pii
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| rubai-ru | 3,458 | 2.89% | 84.47% | — |
| multiconer-ru | 1,208 | 10.76% | 25.41% | 7.86% |
| alexen2 | 1,261 | 11.18% | 93.34% | 16.10% |
| tab-echr | 3,830 | 14.96% | 54.60% | — |
| alrosait | 1,862 | 16.70% | 89.63% | 9.09% |
| nerel | 24,369 | 25.03% | 45.84% | — |
| leak-museum | 101 | 30.69% | 39.60% | 4.25% |
| corrupt-hivetrace | 1,667 | 32.09% | 62.45% | 11.59% |
| corrupt-redmadrobot | 5,531 | 32.98% | 48.54% | 6.29% |
| dialogpii-multi | 10,345 | 34.19% | 51.30% | — |
| jayguard | 1,195 | 38.83% | 64.85% | 4.32% |
| factrueval | 7,966 | 40.16% | 43.80% | — |
| gretel-multi | 4,484 | 40.63% | 78.86% | 2.67% |
| privy | 1,899 | 42.07% | 57.40% | 6.17% |
| ameau01 | 2,242 | 43.00% | 63.83% | 3.19% |
| redmadrobot | 5,516 | 45.25% | 55.71% | 2.89% |
| redact-multi | 22,864 | 46.51% | 71.09% | — |
| scanpatch | 8,708 | 48.85% | 72.85% | 3.79% |
| redact-ru | 9,230 | 50.04% | 74.32% | 9.01% |
| dialogpii-en | 3,087 | 51.15% | 70.65% | 8.76% |
| synth-jira-comments | 2,705 | 52.31% | 74.34% | — |
| hivetrace | 1,667 | 53.27% | 77.32% | 4.57% |
| synth-secrets-ru | 581 | 53.53% | 85.20% | 34.25% |
| synth-env-configs | 392 | 54.08% | 91.33% | 22.75% |
| leaky-repo | 95 | 54.74% | 85.26% | 6.44% |
| synth-ru-tickets | 32,099 | 55.14% | 86.36% | — |
| synth-secrets-en | 581 | 57.14% | 87.09% | 33.81% |
| nym-multi | 8,169 | 61.05% | 71.65% | 4.12% |
| corrupt-secrets-issues | 288 | 62.15% | 90.28% | 11.89% |
| nym-ru | 9,497 | 63.78% | 77.98% | — |
| secrets-rules | 746 | 64.08% | 85.25% | 48.37% |
| synth-wiki-tables | 13,094 | 65.53% | 87.21% | — |
| russian-pii-66k | 4,805 | 66.53% | 79.77% | — |
| tonicai | 2,417 | 67.31% | 71.00% | 0.75% |
| creddata | 776 | 68.30% | 85.70% | 14.16% |
| secrets-issues | 288 | 68.40% | 94.10% | 11.76% |
| kiji-multi | 7,565 | 69.00% | 75.03% | — |
| nym-en | 3,584 | 77.29% | 86.41% | 2.28% |
| arthur-passwords | 280 | 86.07% | 86.07% | 16.53% |
| kiji-en | 7,623 | 89.09% | 91.77% | — |