EN · SECRETS
corrupt-secrets-issues PII detection benchmark results
A frozen evaluation slice with 500 rows, 288 normalized gold annotations and source-level provenance.
500input rows
288gold annotations
1,191,638characters
48measured detectors
Dataset provenance
- Language group
- EN
- Task
- SECRETS
- License
- cc-by-4.0
- Data categories
- SECRET
- Project generated
- No
- Corrupted copy
- Yes
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pii-shield-onnx | ONNX | 84.72% | 100.00% | 53.05% | 0.101 |
| pplx | PPLX | 75.69% | 95.83% | 10.10% | 0.306 |
| pplx+homoglyph | PPLX | 75.69% | 95.83% | 10.28% | 0.304 |
| opf-kz-ru | OPF | 73.26% | 95.49% | 8.23% | 0.319 |
| apararti | OPF | 72.92% | 97.22% | 7.81% | 0.335 |
| opf-ru | OPF | 70.83% | 97.57% | 8.62% | 0.340 |
| bardsai-eu | ONNX | 70.83% | 94.44% | 5.73% | 0.360 |
| opf-ru-v2 | OPF | 70.14% | 93.06% | 3.73% | 0.414 |
| opf-ru-v2+homoglyph | OPF | 70.14% | 92.71% | 3.57% | 0.417 |
| traciora | OPF | 69.44% | 94.44% | 4.55% | 0.404 |
| openai-base | OPF | 68.06% | 89.58% | 4.41% | 0.408 |
| nym-base | HF | 63.19% | 96.18% | 6.33% | 0.356 |
| nym-base+homoglyph | HF | 63.19% | 96.18% | 6.30% | 0.357 |
| openmed-nemotron | HF | 62.15% | 90.28% | 11.89% | 0.273 |
| nym-small | ONNX | 55.90% | 95.83% | 7.31% | 0.329 |
| openmed-multilingual | HF | 53.82% | 89.58% | 9.59% | 0.291 |
| kalyan-ettin | HF | 53.47% | 86.81% | 6.49% | 0.345 |
| gliner-pii-edge | GLINER | 52.78% | 86.46% | 9.04% | 0.332 |
| ru-legal-ner | HF | 41.67% | 77.43% | 11.76% | 0.137 |
| ru-legal-ner+homoglyph | HF | 41.32% | 77.08% | 10.02% | 0.144 |
| nuner-zero | GLINER | 40.97% | 70.49% | 2.23% | 0.442 |
| rules-ru | RULES | 40.97% | 59.72% | 7.35% | 0.196 |
| gliner2-fastino | GLINER2 | 35.76% | 64.58% | 5.76% | 0.332 |
| mmbert32k | HF | 35.07% | 83.33% | 8.84% | 0.305 |
| gliner-urchade | GLINER | 34.38% | 55.21% | 2.86% | 0.246 |
| fef2-secret-ru | HF | 33.33% | 51.04% | 0.43% | 0.546 |
| gliner-nvidia | GLINER | 29.86% | 47.57% | 3.28% | 0.418 |
| gliner-nvidia+homoglyph | GLINER | 29.86% | 47.57% | 3.29% | 0.417 |
| gliner-stream-pii | GLINER | 27.78% | 55.90% | 2.39% | 0.453 |
| gliner25-fastino | GLINER2 | 23.26% | 42.36% | 5.64% | 0.129 |
| stanza-ru | STANZA | 20.83% | 47.22% | 20.76% | 0.144 |
| ner-ru-yqelz | HF | 19.79% | 33.68% | 7.56% | 0.102 |
| ru-pii-ner | RUPII | 19.44% | 26.39% | 1.74% | 0.250 |
| gliner2-hivetrace-omni | GLINER2 | 16.32% | 30.21% | 1.30% | 0.248 |
| gliner2-large | GLINER2 | 11.46% | 29.86% | 2.04% | 0.086 |
| gravitee-small | HF | 11.11% | 27.08% | 3.94% | 0.078 |
| gliner-multi-v21 | GLINER | 8.33% | 26.39% | 1.88% | 0.074 |
| gliner-pii-base | GLINER | 7.64% | 13.54% | 0.56% | 0.155 |
| gliner2-hivetrace-uni | GLINER2 | 5.21% | 14.93% | 0.44% | 0.055 |
| gliner2-vladlinv | GLINER2 | 4.17% | 9.72% | 0.06% | 0.046 |
| gliner2-vladlinv+homoglyph | GLINER2 | 4.17% | 9.72% | 0.04% | 0.047 |
| spacy-ru-lg | SPACY | 2.78% | 5.56% | 3.18% | 0.384 |
| ner-ru-gherman | HF | 0.00% | 0.00% | 0.10% | 0.000 |
| davlan-xlmr | HF | 0.00% | 0.00% | 0.45% | 0.000 |
| davlan-mbert | HF | 0.00% | 1.04% | 0.92% | 0.001 |
| natasha | NATASHA | 0.00% | 2.43% | 0.81% | 0.180 |
| spacy-alrosait | SPACY | 0.00% | 0.00% | 0.02% | 0.000 |
| ner-ru-gherman+homoglyph | HF | 0.00% | 0.00% | 0.12% | 0.000 |