EN · SECRETS
leaky-repo PII detection benchmark results
A frozen evaluation slice with 59 rows, 95 normalized gold annotations and source-level provenance.
59input rows
95gold annotations
62,972characters
45measured detectors
Dataset provenance
- Language group
- EN
- Task
- SECRETS
- License
- mit
- Data categories
- SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pplx | PPLX | 85.26% | 92.63% | 3.32% | 0.713 |
| opf-ru | OPF | 80.00% | 91.58% | 3.32% | 0.868 |
| pii-shield-onnx | ONNX | 76.84% | 82.11% | 30.07% | 0.480 |
| opf-kz-ru | OPF | 75.79% | 83.16% | 3.83% | 0.841 |
| apararti | OPF | 70.53% | 84.21% | 4.00% | 0.850 |
| gliner2-fastino | GLINER2 | 63.16% | 82.11% | 8.41% | 0.510 |
| nym-base | HF | 63.16% | 83.16% | 2.68% | 0.788 |
| traciora | OPF | 61.05% | 81.05% | 1.53% | 0.895 |
| nym-small | ONNX | 60.00% | 88.42% | 6.94% | 0.746 |
| openai-base | OPF | 60.00% | 70.53% | 2.20% | 0.848 |
| openmed-multilingual | HF | 58.95% | 90.53% | 6.82% | 0.739 |
| opf-ru-v2 | OPF | 56.84% | 76.84% | 0.56% | 0.901 |
| openmed-nemotron | HF | 54.74% | 85.26% | 6.44% | 0.735 |
| credsweeper-noml | LEAKS | 54.74% | 55.79% | 0.00% | 0.773 |
| gliner25-fastino | GLINER2 | 53.68% | 64.21% | 16.01% | 0.243 |
| bardsai-eu | ONNX | 53.68% | 78.95% | 2.32% | 0.868 |
| gliner2-hivetrace-omni | GLINER2 | 51.58% | 60.00% | 0.67% | 0.444 |
| nuner-zero | GLINER | 50.53% | 61.05% | 3.08% | 0.354 |
| gliner-pii-edge | GLINER | 50.53% | 77.89% | 6.20% | 0.490 |
| gliner-nvidia | GLINER | 49.47% | 56.84% | 0.59% | 0.393 |
| kalyan-ettin | HF | 49.47% | 76.84% | 2.89% | 0.746 |
| gliner-urchade | GLINER | 42.11% | 51.58% | 5.70% | 0.257 |
| gliner-stream-pii | GLINER | 42.11% | 58.95% | 1.34% | 0.326 |
| mmbert32k | HF | 34.74% | 66.32% | 3.15% | 0.520 |
| fef2-secret-ru | HF | 34.74% | 42.11% | 0.02% | 0.265 |
| rules-ru | RULES | 34.74% | 36.84% | 4.49% | 0.553 |
| ru-legal-ner | HF | 32.63% | 70.53% | 9.14% | 0.576 |
| gliner2-large | GLINER2 | 32.63% | 41.05% | 3.45% | 0.164 |
| detect-secrets | LEAKS | 32.63% | 34.74% | 0.00% | 0.241 |
| deepsecrets | LEAKS | 28.42% | 30.53% | 0.00% | 0.705 |
| gravitee-small | HF | 25.26% | 42.11% | 1.96% | 0.207 |
| gitleaks | LEAKS | 23.16% | 23.16% | 0.00% | 0.705 |
| gliner-multi-v21 | GLINER | 22.11% | 30.53% | 1.58% | 0.089 |
| gliner2-vladlinv | GLINER2 | 17.89% | 18.95% | 0.00% | 0.143 |
| gliner2-hivetrace-uni | GLINER2 | 16.84% | 18.95% | 0.00% | 0.079 |
| stanza-ru | STANZA | 14.74% | 25.26% | 11.30% | 0.369 |
| ner-ru-yqelz | HF | 13.68% | 18.95% | 14.58% | 0.065 |
| gliner-pii-base | GLINER | 11.58% | 14.74% | 0.62% | 0.050 |
| ru-pii-ner | RUPII | 5.26% | 11.58% | 0.09% | 0.149 |
| spacy-ru-lg | SPACY | 4.21% | 4.21% | 1.03% | 0.022 |
| ner-ru-gherman | HF | 0.00% | 0.00% | 0.03% | 0.000 |
| davlan-xlmr | HF | 0.00% | 0.00% | 0.42% | 0.000 |
| davlan-mbert | HF | 0.00% | 0.00% | 0.67% | 0.000 |
| natasha | NATASHA | 0.00% | 1.05% | 0.61% | 0.000 |
| spacy-alrosait | SPACY | 0.00% | 0.00% | 0.00% | 0.000 |