EN · SECRETS
synth-secrets-en PII detection benchmark results
A frozen evaluation slice with 600 rows, 581 normalized gold annotations and source-level provenance.
600input rows
581gold annotations
273,798characters
41measured detectors
Dataset provenance
- Language group
- EN
- Task
- SECRETS
- License
- cc-by-4.0
- Data categories
- SECRET
- Project generated
- Yes
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pii-shield-onnx | ONNX | 97.76% | 99.66% | 70.06% | 0.241 |
| pplx | PPLX | 97.42% | 99.83% | 43.92% | 0.330 |
| opf-kz-ru | OPF | 91.39% | 98.28% | 47.61% | 0.319 |
| apararti | OPF | 87.95% | 97.07% | 50.54% | 0.304 |
| openai-base | OPF | 84.51% | 92.94% | 49.46% | 0.300 |
| openmed-multilingual | HF | 82.44% | 99.14% | 41.95% | 0.335 |
| rules-ru | RULES | 81.76% | 82.27% | 16.69% | 0.519 |
| opf-ru-v2 | OPF | 78.66% | 96.04% | 32.22% | 0.378 |
| opf-ru | OPF | 77.45% | 98.28% | 29.14% | 0.404 |
| bardsai-eu | ONNX | 73.49% | 92.94% | 27.99% | 0.398 |
| traciora | OPF | 70.57% | 94.49% | 30.64% | 0.396 |
| gliner-pii-edge | GLINER | 70.57% | 83.65% | 24.36% | 0.400 |
| nym-base | HF | 62.48% | 93.98% | 32.00% | 0.354 |
| gliner-nvidia | GLINER | 61.96% | 64.37% | 11.22% | 0.473 |
| gliner2-fastino | GLINER2 | 58.00% | 64.54% | 4.59% | 0.466 |
| nym-small | ONNX | 57.31% | 93.29% | 30.86% | 0.352 |
| openmed-nemotron | HF | 57.14% | 87.09% | 33.81% | 0.353 |
| gliner2-hivetrace-omni | GLINER2 | 54.22% | 54.56% | 6.50% | 0.422 |
| gliner25-fastino | GLINER2 | 54.04% | 56.80% | 22.91% | 0.261 |
| nuner-zero | GLINER | 53.18% | 64.37% | 4.00% | 0.483 |
| fef2-secret-ru | HF | 53.18% | 71.08% | 4.40% | 0.574 |
| kalyan-ettin | HF | 48.02% | 85.54% | 25.69% | 0.379 |
| gravitee-small | HF | 46.30% | 70.91% | 11.10% | 0.356 |
| gliner-stream-pii | GLINER | 41.31% | 48.54% | 4.13% | 0.390 |
| ru-legal-ner | HF | 33.56% | 89.16% | 27.21% | 0.368 |
| gliner-pii-base | GLINER | 31.15% | 34.60% | 2.63% | 0.319 |
| gliner-urchade | GLINER | 27.88% | 30.64% | 1.31% | 0.258 |
| gliner2-vladlinv | GLINER2 | 25.13% | 27.02% | 3.28% | 0.189 |
| mmbert32k | HF | 20.83% | 78.31% | 33.00% | 0.255 |
| stanza-ru | STANZA | 20.65% | 45.09% | 15.70% | 0.227 |
| gliner-multi-v21 | GLINER | 18.24% | 25.13% | 1.95% | 0.131 |
| gliner2-large | GLINER2 | 17.73% | 19.45% | 2.10% | 0.141 |
| ru-pii-ner | RUPII | 10.67% | 14.63% | 4.62% | 0.111 |
| ner-ru-yqelz | HF | 5.68% | 11.36% | 14.06% | 0.080 |
| gliner2-hivetrace-uni | GLINER2 | 0.86% | 0.86% | 0.09% | 0.006 |
| ner-ru-gherman | HF | 0.00% | 0.00% | 0.01% | 0.000 |
| spacy-ru-lg | SPACY | 0.00% | 0.00% | 0.78% | 0.000 |
| davlan-xlmr | HF | 0.00% | 0.00% | 0.00% | 0.000 |
| davlan-mbert | HF | 0.00% | 0.00% | 0.01% | 0.000 |
| natasha | NATASHA | 0.00% | 0.00% | 0.35% | 0.000 |
| spacy-alrosait | SPACY | 0.00% | 0.00% | 0.00% | 0.000 |