HF detector
kalyan-ettin+cpu-int8 PII detection benchmark
Measured on 8 of 41 retained datasets under the same frozen scoring and masking protocol.
20.89%fully hidden annotations
36.52%detected / overlapped
2.63%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Logins & usernames57.91%4,958 / 8,562
People's names38.95%3,425 / 8,794
Addresses & locations31.07%2,311 / 7,439
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers0.07%13 / 18,140
Organizations3.11%116 / 3,725
Customer & employee IDs3.26%46 / 1,410
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 3 | 13.32% | 37.72% | 3.37% |
| Russian | 4 | 18.67% | 34.52% | 0.60% |
| Multilingual | 1 | 41.57% | 48.39% | — |
Configuration, source and version
- Family
- HF
- Revision
- 500262a2aaf913825ef750ef255c3fe437cd8e64
- Eligible datasets
- 8 / 41
- Character F1
- 0.344
- Untouched diagnostic
- 63.48% · 39,806 / 62,704
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- Not measured
source overlap: nemotron-pii
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| tab-echr | 3,830 | 7.81% | 34.60% | — |
| redmadrobot | 5,516 | 17.46% | 27.30% | 0.58% |
| ameau01 | 2,242 | 17.62% | 37.78% | 1.37% |
| synth-ru-tickets | 32,099 | 17.72% | 34.14% | — |
| nym-ru | 9,497 | 20.26% | 37.37% | — |
| hivetrace | 1,667 | 32.03% | 49.43% | 0.75% |
| kiji-multi | 7,565 | 41.57% | 48.39% | — |
| secrets-issues | 288 | 53.13% | 78.82% | 3.75% |