ONNX detector
openai-base-onnx PII detection benchmark
Measured on 9 of 41 retained datasets under the same frozen scoring and masking protocol.
70.86%fully hidden annotations
76.66%detected / overlapped
3.20%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Dates & times96.29%1,792 / 1,861
Phone numbers & email93.44%6,063 / 6,489
Bank accounts & cards86.17%1,620 / 1,880
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Other sensitive attributes5.36%12 / 224
Organizations9.79%447 / 4,568
Addresses & locations53.79%4,791 / 8,907
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 4 | 55.31% | 57.09% | 3.84% |
| Russian | 4 | 74.97% | 82.76% | 1.42% |
| Multilingual | 1 | 76.75% | 78.02% | — |
Configuration, source and version
- Family
- ONNX
- Revision
- 9b7e2f5c794da2c156bd9583bf8e2f5f6bdfd18b
- Eligible datasets
- 9 / 41
- Character F1
- 0.713
- Untouched diagnostic
- 23.34% · 16,829 / 72,095
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- Not measured
mirror of openai-base
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| tab-echr | 3,830 | 22.35% | 23.16% | — |
| ameau01 | 2,242 | 53.30% | 54.86% | 2.58% |
| redmadrobot | 5,516 | 65.19% | 70.43% | 1.39% |
| nemotron-pii | 9,391 | 68.76% | 70.38% | 2.55% |
| secrets-issues | 288 | 70.83% | 92.71% | 4.09% |
| synth-ru-tickets | 32,099 | 73.82% | 83.05% | — |
| kiji-multi | 7,565 | 76.75% | 78.02% | — |
| nym-ru | 9,497 | 82.43% | 87.68% | — |
| hivetrace | 1,667 | 86.98% | 90.10% | 1.66% |