HF detector
fef2-secret-ru+cpu-int8 PII detection benchmark
Measured on 9 of 41 retained datasets under the same frozen scoring and masking protocol.
27.12%fully hidden annotations
34.61%detected / overlapped
0.50%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Logins & usernames84.51%7,429 / 8,791
Passwords, keys & tokens46.91%790 / 1,684
People's names46.10%5,089 / 11,039
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers0.05%9 / 18,902
Dates & times0.48%9 / 1,861
Other sensitive attributes2.68%6 / 224
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 4 | 9.59% | 14.74% | 0.35% |
| Russian | 4 | 36.44% | 45.66% | 0.92% |
| Multilingual | 1 | 3.58% | 4.72% | — |
Configuration, source and version
- Family
- HF
- Revision
- 52b5b0745aac14f73fcf2ac0f91d9b5001a85ae4
- Eligible datasets
- 9 / 41
- Character F1
- 0.443
- Untouched diagnostic
- 65.39% · 47,144 / 72,095
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- Not measured
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| tab-echr | 3,830 | 2.06% | 10.31% | — |
| kiji-multi | 7,565 | 3.58% | 4.72% | — |
| ameau01 | 2,242 | 10.04% | 12.62% | 0.87% |
| nemotron-pii | 9,391 | 11.89% | 15.97% | 0.00% |
| secrets-issues | 288 | 30.90% | 49.65% | 0.25% |
| synth-ru-tickets | 32,099 | 32.06% | 41.53% | — |
| nym-ru | 9,497 | 42.86% | 52.16% | — |
| hivetrace | 1,667 | 46.49% | 58.61% | 0.20% |
| redmadrobot | 5,516 | 47.81% | 54.59% | 1.03% |