HF detector
nym-base+sent300 PII detection benchmark
Measured on 11 of 41 retained datasets under the same frozen scoring and masking protocol.
67.31%fully hidden annotations
84.71%detected / overlapped
8.02%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Logins & usernames99.87%7,784 / 7,794
Dates & times94.18%566 / 601
Bank accounts & cards92.01%702 / 763
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Other sensitive attributes31.72%282 / 889
Network identifiers39.81%6,966 / 17,498
Customer & employee IDs54.33%1,285 / 2,365
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 6 | 69.35% | 85.61% | 8.80% |
| Russian | 5 | 66.84% | 84.51% | 2.92% |
Configuration, source and version
- Family
- HF
- Revision
- 0ed0242d67a30ca6a83e5f118506ff1e1daf83ca
- Eligible datasets
- 11 / 41
- Character F1
- 0.670
- Untouched diagnostic
- 15.29% · 14,114 / 92,332
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- 0.137
source overlap: nym-ru, nym-en, nym-multi
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| leak-museum | 101 | 35.64% | 48.51% | 3.78% |
| tab-echr | 3,830 | 39.32% | 73.79% | — |
| jayguard | 1,195 | 56.32% | 86.78% | 2.75% |
| dialogpii-en | 3,087 | 58.31% | 77.49% | 3.38% |
| synth-ru-tickets | 32,099 | 58.50% | 81.49% | — |
| redact-ru | 9,230 | 61.37% | 85.66% | 9.54% |
| secrets-issues | 288 | 71.53% | 98.96% | 8.05% |
| nerel | 24,369 | 74.47% | 87.24% | — |
| nemotron-pii | 9,391 | 84.04% | 91.99% | 5.53% |
| factrueval | 7,966 | 85.00% | 86.59% | — |
| creddata | 776 | 87.24% | 98.84% | 12.92% |