HF detector
nym-base+cpu-int8 PII detection benchmark
Measured on 8 of 41 retained datasets under the same frozen scoring and masking protocol.
59.79%fully hidden annotations
74.91%detected / overlapped
5.04%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Logins & usernames96.97%8,219 / 8,476
Dates & times95.96%404 / 421
Bank accounts & cards95.09%1,221 / 1,284
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers17.03%3,186 / 18,708
Organizations54.67%2,249 / 4,114
People's names60.80%5,474 / 9,003
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 4 | 65.39% | 82.02% | 6.28% |
| Russian | 3 | 50.89% | 67.88% | 1.59% |
| Multilingual | 1 | 94.38% | 96.59% | — |
Configuration, source and version
- Family
- HF
- Revision
- 0ed0242d67a30ca6a83e5f118506ff1e1daf83ca
- Eligible datasets
- 8 / 41
- Character F1
- 0.607
- Untouched diagnostic
- 25.09% · 15,707 / 62,598
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- Not measured
source overlap: nym-ru, nym-en, nym-multi
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| ameau01 | 2,242 | 39.56% | 58.16% | 1.59% |
| tab-echr | 3,830 | 41.49% | 74.83% | — |
| synth-ru-tickets | 32,099 | 45.26% | 63.33% | — |
| secrets-issues | 288 | 69.44% | 98.26% | 7.19% |
| hivetrace | 1,667 | 74.75% | 98.68% | 0.36% |
| redmadrobot | 5,516 | 76.40% | 85.06% | 1.78% |
| nemotron-pii | 9,391 | 81.18% | 90.15% | 5.22% |
| kiji-multi | 7,565 | 94.38% | 96.59% | — |