ONNX detector
pii-shield-onnx PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
56.43%fully hidden annotations
69.64%detected / overlapped
38.92%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Passwords, keys & tokens88.40%6,073 / 6,870
Phone numbers & email82.13%18,053 / 21,981
Logins & usernames81.68%9,297 / 11,382
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Organizations16.48%3,177 / 19,274
Other sensitive attributes41.47%1,330 / 3,207
Addresses & locations44.34%17,560 / 39,602
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 40.53% | 49.99% | 44.87% |
| Russian | 19 | 56.17% | 71.66% | 26.09% |
| Multilingual | 5 | 68.32% | 78.31% | 14.94% |
Configuration, source and version
- Family
- ONNX
- Revision
- 280f050f10a73db88fa9b864e8ca25350e73941e
- Eligible datasets
- 41 / 41
- Character F1
- 0.416
- Untouched diagnostic
- 30.36% · 69,057 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- Not measured
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| rubai-ru | 3,458 | 5.12% | 88.29% | — |
| multiconer-ru | 1,208 | 11.18% | 22.85% | 4.86% |
| tab-echr | 3,830 | 12.40% | 30.60% | — |
| dialogpii-en | 3,087 | 17.65% | 27.37% | 7.48% |
| alexen2 | 1,261 | 22.84% | 95.00% | 9.40% |
| tonicai | 2,417 | 25.69% | 27.84% | 0.56% |
| nerel | 24,369 | 31.05% | 39.48% | — |
| gretel-multi | 4,484 | 34.72% | 53.88% | 15.13% |
| kiji-en | 7,623 | 37.58% | 43.46% | — |
| nym-en | 3,584 | 39.23% | 52.06% | 2.69% |
| factrueval | 7,966 | 39.88% | 42.02% | — |
| nemotron-pii | 9,391 | 39.93% | 50.78% | 9.96% |
| ameau01 | 2,242 | 46.79% | 58.83% | 6.95% |
| jayguard | 1,195 | 47.70% | 68.28% | 6.02% |
| alrosait | 1,862 | 52.74% | 87.49% | 3.55% |
| corrupt-redmadrobot | 5,531 | 54.40% | 67.85% | 44.25% |
| redact-ru | 9,230 | 55.51% | 70.10% | 6.61% |
| scanpatch | 8,708 | 58.23% | 77.45% | 1.56% |
| nym-ru | 9,497 | 58.92% | 72.92% | — |
| redmadrobot | 5,516 | 59.05% | 67.95% | 10.35% |
| kiji-multi | 7,565 | 62.49% | 68.16% | — |
| synth-wiki-tables | 13,094 | 63.45% | 77.07% | — |
| privy | 1,899 | 63.93% | 72.14% | 56.48% |
| redact-multi | 22,864 | 70.23% | 81.56% | — |
| corrupt-hivetrace | 1,667 | 71.33% | 87.88% | 15.27% |
| dialogpii-multi | 10,345 | 74.31% | 82.24% | — |
| synth-ru-tickets | 32,099 | 75.35% | 93.37% | — |
| leaky-repo | 95 | 76.84% | 82.11% | 30.07% |
| hivetrace | 1,667 | 78.82% | 89.74% | 0.28% |
| russian-pii-66k | 4,805 | 78.88% | 85.64% | — |
| nym-multi | 8,169 | 79.21% | 87.02% | 1.87% |
| leak-museum | 101 | 83.17% | 90.10% | 9.91% |
| secrets-issues | 288 | 83.68% | 99.65% | 38.24% |
| corrupt-secrets-issues | 288 | 84.72% | 100.00% | 53.05% |
| synth-jira-comments | 2,705 | 85.62% | 91.20% | — |
| creddata | 776 | 92.14% | 96.26% | 40.24% |
| synth-secrets-ru | 581 | 97.42% | 99.66% | 70.31% |
| synth-secrets-en | 581 | 97.76% | 99.66% | 70.06% |
| arthur-passwords | 280 | 97.86% | 97.86% | 13.24% |
| secrets-rules | 746 | 98.53% | 99.87% | 80.36% |
| synth-env-configs | 392 | 99.23% | 100.00% | 76.71% |