EN · SECRETS
leaky-repo PII detection benchmark results
A frozen evaluation slice with 59 rows, 95 normalized gold annotations and source-level provenance.
59input rows
95gold annotations
62,972characters
45measured detectors
Dataset provenance
- Language group
- EN
- Task
- SECRETS
- License
- mit
- Data categories
- SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by untouched annotations. Training-source overlaps are excluded.
| Detector | Family | Untouched ↓ | Fully hidden ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pplx | PPLX | 7.37% | 85.26% | 3.32% | 0.713 |
| opf-ru | OPF | 8.42% | 80.00% | 3.32% | 0.868 |
| openmed-multilingual | HF | 9.47% | 58.95% | 6.82% | 0.739 |
| nym-small | ONNX | 11.58% | 60.00% | 6.94% | 0.746 |
| openmed-nemotron | HF | 14.74% | 54.74% | 6.44% | 0.735 |
| apararti | OPF | 15.79% | 70.53% | 4.00% | 0.850 |
| nym-base | HF | 16.84% | 63.16% | 2.68% | 0.788 |
| opf-kz-ru | OPF | 16.84% | 75.79% | 3.83% | 0.841 |
| gliner2-fastino | GLINER2 | 17.89% | 63.16% | 8.41% | 0.510 |
| pii-shield-onnx | ONNX | 17.89% | 76.84% | 30.07% | 0.480 |
| traciora | OPF | 18.95% | 61.05% | 1.53% | 0.895 |
| bardsai-eu | ONNX | 21.05% | 53.68% | 2.32% | 0.868 |
| gliner-pii-edge | GLINER | 22.11% | 50.53% | 6.20% | 0.490 |
| opf-ru-v2 | OPF | 23.16% | 56.84% | 0.56% | 0.901 |
| kalyan-ettin | HF | 23.16% | 49.47% | 2.89% | 0.746 |
| ru-legal-ner | HF | 29.47% | 32.63% | 9.14% | 0.576 |
| openai-base | OPF | 29.47% | 60.00% | 2.20% | 0.848 |
| mmbert32k | HF | 33.68% | 34.74% | 3.15% | 0.520 |
| gliner25-fastino | GLINER2 | 35.79% | 53.68% | 16.01% | 0.243 |
| nuner-zero | GLINER | 38.95% | 50.53% | 3.08% | 0.354 |
| gliner2-hivetrace-omni | GLINER2 | 40.00% | 51.58% | 0.67% | 0.444 |
| gliner-stream-pii | GLINER | 41.05% | 42.11% | 1.34% | 0.326 |
| gliner-nvidia | GLINER | 43.16% | 49.47% | 0.59% | 0.393 |
| credsweeper-noml | LEAKS | 44.21% | 54.74% | 0.00% | 0.773 |
| gliner-urchade | GLINER | 48.42% | 42.11% | 5.70% | 0.257 |
| gravitee-small | HF | 57.89% | 25.26% | 1.96% | 0.207 |
| fef2-secret-ru | HF | 57.89% | 34.74% | 0.02% | 0.265 |
| gliner2-large | GLINER2 | 58.95% | 32.63% | 3.45% | 0.164 |
| rules-ru | RULES | 63.16% | 34.74% | 4.49% | 0.553 |
| detect-secrets | LEAKS | 65.26% | 32.63% | 0.00% | 0.241 |
| gliner-multi-v21 | GLINER | 69.47% | 22.11% | 1.58% | 0.089 |
| deepsecrets | LEAKS | 69.47% | 28.42% | 0.00% | 0.705 |
| stanza-ru | STANZA | 74.74% | 14.74% | 11.30% | 0.369 |
| gitleaks | LEAKS | 76.84% | 23.16% | 0.00% | 0.705 |
| gliner2-vladlinv | GLINER2 | 81.05% | 17.89% | 0.00% | 0.143 |
| gliner2-hivetrace-uni | GLINER2 | 81.05% | 16.84% | 0.00% | 0.079 |
| ner-ru-yqelz | HF | 81.05% | 13.68% | 14.58% | 0.065 |
| gliner-pii-base | GLINER | 85.26% | 11.58% | 0.62% | 0.050 |
| ru-pii-ner | RUPII | 88.42% | 5.26% | 0.09% | 0.149 |
| spacy-ru-lg | SPACY | 95.79% | 4.21% | 1.03% | 0.022 |
| natasha | NATASHA | 98.95% | 0.00% | 0.61% | 0.000 |
| ner-ru-gherman | HF | 100.00% | 0.00% | 0.03% | 0.000 |
| davlan-xlmr | HF | 100.00% | 0.00% | 0.42% | 0.000 |
| davlan-mbert | HF | 100.00% | 0.00% | 0.67% | 0.000 |
| spacy-alrosait | SPACY | 100.00% | 0.00% | 0.00% | 0.000 |