HF detector
nym-base+ov100 PII detection benchmark
Measured on 11 of 41 retained datasets under the same frozen scoring and masking protocol.
63.27%fully hidden annotations
76.72%detected / overlapped
6.73%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Logins & usernames99.37%7,745 / 7,794
Dates & times93.68%563 / 601
Bank accounts & cards91.48%698 / 763
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers7.34%1,284 / 17,498
Other sensitive attributes32.51%289 / 889
Customer & employee IDs53.78%1,272 / 2,365
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 6 | 69.84% | 85.18% | 7.32% |
| Russian | 5 | 61.74% | 74.75% | 2.87% |
Configuration, source and version
- Family
- HF
- Revision
- 0ed0242d67a30ca6a83e5f118506ff1e1daf83ca
- Eligible datasets
- 11 / 41
- Character F1
- 0.620
- Untouched diagnostic
- 23.28% · 21,493 / 92,332
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- 0.140
source overlap: nym-ru, nym-en, nym-multi
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| leak-museum | 101 | 34.65% | 46.53% | 3.16% |
| synth-ru-tickets | 32,099 | 40.70% | 54.80% | — |
| tab-echr | 3,830 | 46.16% | 76.97% | — |
| jayguard | 1,195 | 56.40% | 86.86% | 2.69% |
| dialogpii-en | 3,087 | 56.59% | 75.57% | 3.31% |
| redact-ru | 9,230 | 60.85% | 84.40% | 9.99% |
| secrets-issues | 288 | 70.49% | 99.31% | 6.65% |
| nerel | 24,369 | 81.46% | 91.96% | — |
| nemotron-pii | 9,391 | 82.76% | 90.54% | 5.22% |
| creddata | 776 | 87.37% | 98.71% | 10.91% |
| factrueval | 7,966 | 88.02% | 89.47% | — |