OPF detector
traciora PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
48.60%fully hidden annotations
59.54%detected / overlapped
7.51%extra masking
3,040CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Phone numbers & email85.85%18,871 / 21,981
Passwords, keys & tokens80.36%5,521 / 6,870
People's names66.70%43,845 / 65,736
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Dates & times2.34%129 / 5,502
Other sensitive attributes9.51%305 / 3,207
Organizations9.87%1,902 / 19,274
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 54.65% | 60.92% | 7.86% |
| Russian | 19 | 46.10% | 59.45% | 7.02% |
| Multilingual | 5 | 50.72% | 58.81% | 0.10% |
Configuration, source and version
- Family
- OPF
- Revision
- ea69da97d598b662ec447bcb8e0022abee0f4010
- Eligible datasets
- 41 / 41
- Character F1
- 0.638
- Untouched diagnostic
- 40.46% · 92,030 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 3.289
- GPU seconds / 10k chars
- 1.359
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| rubai-ru | 3,458 | 7.11% | 95.89% | — |
| multiconer-ru | 1,208 | 13.74% | 25.75% | 4.95% |
| tab-echr | 3,830 | 16.95% | 20.63% | — |
| nerel | 24,369 | 24.74% | 29.27% | — |
| alexen2 | 1,261 | 25.85% | 96.75% | 12.30% |
| factrueval | 7,966 | 31.02% | 32.05% | — |
| synth-ru-tickets | 32,099 | 35.74% | 55.52% | — |
| dialogpii-multi | 10,345 | 36.49% | 45.79% | — |
| dialogpii-en | 3,087 | 40.30% | 46.39% | 4.43% |
| ameau01 | 2,242 | 44.78% | 49.60% | 1.29% |
| tonicai | 2,417 | 48.99% | 52.05% | 0.23% |
| nym-multi | 8,169 | 49.27% | 57.78% | 0.00% |
| privy | 1,899 | 49.82% | 59.87% | 8.04% |
| corrupt-redmadrobot | 5,531 | 50.84% | 65.34% | 2.56% |
| nym-en | 3,584 | 51.95% | 59.12% | 1.34% |
| redact-multi | 22,864 | 52.29% | 60.69% | — |
| nemotron-pii | 9,391 | 52.49% | 59.35% | 0.00% |
| jayguard | 1,195 | 53.72% | 66.44% | 1.85% |
| gretel-multi | 4,484 | 54.35% | 64.47% | 0.10% |
| synth-jira-comments | 2,705 | 56.67% | 64.14% | — |
| scanpatch | 8,708 | 59.18% | 67.50% | 3.09% |
| redmadrobot | 5,516 | 59.28% | 68.84% | 1.15% |
| nym-ru | 9,497 | 60.56% | 71.30% | — |
| leaky-repo | 95 | 61.05% | 81.05% | 1.53% |
| redact-ru | 9,230 | 62.62% | 70.18% | 5.92% |
| leak-museum | 101 | 63.37% | 71.29% | 3.16% |
| kiji-multi | 7,565 | 64.85% | 68.66% | — |
| synth-wiki-tables | 13,094 | 65.33% | 77.32% | — |
| synth-env-configs | 392 | 67.09% | 99.49% | 21.28% |
| corrupt-secrets-issues | 288 | 69.44% | 94.44% | 4.55% |
| synth-secrets-en | 581 | 70.57% | 94.49% | 30.64% |
| secrets-issues | 288 | 71.88% | 96.88% | 3.73% |
| corrupt-hivetrace | 1,667 | 74.21% | 89.08% | 11.17% |
| kiji-en | 7,623 | 76.64% | 79.92% | — |
| alrosait | 1,862 | 76.80% | 92.27% | 6.91% |
| synth-secrets-ru | 581 | 77.28% | 97.07% | 32.45% |
| hivetrace | 1,667 | 85.66% | 89.80% | 1.57% |
| russian-pii-66k | 4,805 | 86.24% | 89.28% | — |
| creddata | 776 | 91.37% | 97.55% | 4.08% |
| arthur-passwords | 280 | 93.57% | 93.57% | 11.49% |
| secrets-rules | 746 | 97.45% | 99.87% | 48.01% |