OPF detector
openai-base PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
59.69%fully hidden annotations
65.39%detected / overlapped
8.83%extra masking
2,378CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Phone numbers & email89.33%19,635 / 21,981
Dates & times87.71%4,826 / 5,502
Passwords, keys & tokens85.56%5,878 / 6,870
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Organizations6.16%1,188 / 19,274
Other sensitive attributes11.44%367 / 3,207
Addresses & locations39.64%15,698 / 39,602
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 66.96% | 69.46% | 8.90% |
| Russian | 19 | 56.63% | 63.95% | 9.00% |
| Multilingual | 5 | 62.39% | 66.21% | 1.46% |
Configuration, source and version
- Family
- OPF
- Revision
- 7ffa9a043d54d1be65afb281eddf0ffbe629385b
- Eligible datasets
- 41 / 41
- Character F1
- 0.657
- Untouched diagnostic
- 34.61% · 78,725 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 4.205
- GPU seconds / 10k chars
- 1.804
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| multiconer-ru | 1,208 | 4.22% | 7.12% | 1.33% |
| nerel | 24,369 | 8.88% | 9.70% | — |
| factrueval | 7,966 | 14.02% | 14.16% | — |
| alexen2 | 1,261 | 20.86% | 87.87% | 8.63% |
| tab-echr | 3,830 | 22.22% | 23.13% | — |
| rubai-ru | 3,458 | 26.17% | 93.18% | — |
| dialogpii-multi | 10,345 | 31.43% | 35.87% | — |
| jayguard | 1,195 | 46.03% | 54.98% | 1.85% |
| corrupt-redmadrobot | 5,531 | 51.82% | 59.36% | 1.79% |
| ameau01 | 2,242 | 53.70% | 55.04% | 2.58% |
| dialogpii-en | 3,087 | 54.13% | 57.60% | 12.89% |
| leak-museum | 101 | 56.44% | 63.37% | 1.15% |
| leaky-repo | 95 | 60.00% | 70.53% | 2.20% |
| synth-jira-comments | 2,705 | 60.78% | 66.77% | — |
| privy | 1,899 | 63.82% | 68.83% | 8.68% |
| gretel-multi | 4,484 | 64.74% | 71.19% | 1.44% |
| redmadrobot | 5,516 | 65.17% | 70.58% | 1.39% |
| tonicai | 2,417 | 65.37% | 68.51% | 0.30% |
| corrupt-secrets-issues | 288 | 68.06% | 89.58% | 4.41% |
| redact-multi | 22,864 | 68.17% | 72.01% | — |
| corrupt-hivetrace | 1,667 | 68.63% | 77.50% | 2.48% |
| nemotron-pii | 9,391 | 68.64% | 70.27% | 2.55% |
| scanpatch | 8,708 | 70.22% | 74.39% | 1.30% |
| nym-multi | 8,169 | 70.94% | 74.88% | 3.18% |
| secrets-issues | 288 | 71.18% | 92.71% | 4.09% |
| synth-ru-tickets | 32,099 | 73.68% | 82.96% | — |
| alrosait | 1,862 | 73.85% | 87.16% | 5.39% |
| kiji-multi | 7,565 | 76.66% | 77.83% | — |
| redact-ru | 9,230 | 76.94% | 80.66% | 11.76% |
| synth-env-configs | 392 | 77.30% | 95.15% | 14.20% |
| nym-en | 3,584 | 78.43% | 81.78% | 1.91% |
| synth-wiki-tables | 13,094 | 80.89% | 87.03% | — |
| nym-ru | 9,497 | 82.37% | 87.67% | — |
| kiji-en | 7,623 | 84.36% | 84.81% | — |
| synth-secrets-en | 581 | 84.51% | 92.94% | 49.46% |
| hivetrace | 1,667 | 86.86% | 90.10% | 1.70% |
| creddata | 776 | 88.40% | 91.11% | 4.64% |
| russian-pii-66k | 4,805 | 89.82% | 92.20% | — |
| synth-secrets-ru | 581 | 91.74% | 97.93% | 51.56% |
| arthur-passwords | 280 | 92.86% | 92.86% | 13.82% |
| secrets-rules | 746 | 97.99% | 99.06% | 45.69% |