ONNX detector
nym-small PII detection benchmark
Measured on 38 of 41 retained datasets under the same frozen scoring and masking protocol.
62.82%fully hidden annotations
81.01%detected / overlapped
9.82%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Dates & times90.00%2,161 / 2,401
Logins & usernames85.72%9,273 / 10,818
Phone numbers & email82.37%16,274 / 19,756
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Network identifiers22.87%5,792 / 25,323
Other sensitive attributes28.96%819 / 2,828
Customer & employee IDs45.83%3,127 / 6,823
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 16 | 73.44% | 86.02% | 11.33% |
| Russian | 18 | 60.40% | 80.00% | 6.67% |
| Multilingual | 4 | 61.61% | 80.07% | 1.68% |
Configuration, source and version
- Family
- ONNX
- Revision
- 4348999cd3c2e20c49615e9af7c6bbb45b64cd85
- Eligible datasets
- 38 / 41
- Character F1
- 0.673
- Untouched diagnostic
- 18.99% · 39,158 / 206,216
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- Not measured
source overlap: nym-ru, nym-en, nym-multi
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| rubai-ru | 3,458 | 5.00% | 98.26% | — |
| alexen2 | 1,261 | 10.23% | 96.99% | 12.49% |
| alrosait | 1,862 | 17.83% | 98.50% | 4.65% |
| multiconer-ru | 1,208 | 20.03% | 29.88% | 5.03% |
| synth-jira-comments | 2,705 | 35.97% | 69.94% | — |
| synth-ru-tickets | 32,099 | 38.40% | 55.48% | — |
| ameau01 | 2,242 | 38.40% | 55.66% | 1.85% |
| leak-museum | 101 | 38.61% | 48.51% | 4.00% |
| tab-echr | 3,830 | 44.26% | 77.13% | — |
| gretel-multi | 4,484 | 46.83% | 82.67% | 1.68% |
| jayguard | 1,195 | 48.79% | 78.49% | 2.66% |
| dialogpii-multi | 10,345 | 51.45% | 67.96% | — |
| synth-env-configs | 392 | 51.53% | 97.19% | 28.19% |
| corrupt-secrets-issues | 288 | 55.90% | 95.83% | 7.31% |
| synth-secrets-en | 581 | 57.31% | 93.29% | 30.86% |
| redact-multi | 22,864 | 57.85% | 79.33% | — |
| dialogpii-en | 3,087 | 58.28% | 77.97% | 3.83% |
| redact-ru | 9,230 | 59.90% | 82.77% | 8.67% |
| leaky-repo | 95 | 60.00% | 88.42% | 6.94% |
| synth-secrets-ru | 581 | 60.93% | 94.49% | 32.58% |
| tonicai | 2,417 | 62.02% | 64.75% | 0.28% |
| corrupt-redmadrobot | 5,531 | 62.34% | 78.47% | 2.21% |
| corrupt-hivetrace | 1,667 | 62.87% | 95.08% | 1.68% |
| secrets-issues | 288 | 67.01% | 97.92% | 7.56% |
| scanpatch | 8,708 | 71.30% | 89.62% | 1.33% |
| synth-wiki-tables | 13,094 | 74.50% | 96.27% | — |
| redmadrobot | 5,516 | 75.49% | 84.95% | 2.03% |
| hivetrace | 1,667 | 75.94% | 99.10% | 0.34% |
| nerel | 24,369 | 76.38% | 88.47% | — |
| secrets-rules | 746 | 79.09% | 97.32% | 46.28% |
| privy | 1,899 | 79.09% | 93.26% | 11.13% |
| nemotron-pii | 9,391 | 83.19% | 90.55% | 5.22% |
| creddata | 776 | 83.76% | 98.07% | 14.27% |
| factrueval | 7,966 | 84.70% | 86.32% | — |
| kiji-multi | 7,565 | 95.64% | 97.30% | — |
| kiji-en | 7,623 | 96.09% | 97.74% | — |
| arthur-passwords | 280 | 99.29% | 99.29% | 16.60% |
| russian-pii-66k | 4,805 | 99.42% | 99.71% | — |