EN · SECRETS
leak-museum PII detection benchmark results
A frozen evaluation slice with 97 rows, 101 normalized gold annotations and source-level provenance.
97input rows
101gold annotations
75,730characters
55measured detectors
Dataset provenance
- Language group
- EN
- Task
- SECRETS
- License
- mit
- Data categories
- SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by untouched annotations. Training-source overlaps are excluded.
| Detector | Family | Untouched ↓ | Fully hidden ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pii-shield-onnx | ONNX | 9.90% | 83.17% | 9.91% | 0.381 |
| pplx+ov100 | PPLX | 11.88% | 86.14% | 3.90% | 0.643 |
| opf-ru | OPF | 14.85% | 77.23% | 4.79% | 0.668 |
| pplx | PPLX | 16.83% | 82.18% | 4.73% | 0.604 |
| pplx+sent300 | PPLX | 16.83% | 82.18% | 3.45% | 0.623 |
| gliner2-fastino | GLINER2 | 19.80% | 72.28% | 10.94% | 0.414 |
| gliner-pii-edge | GLINER | 24.75% | 59.41% | 7.23% | 0.446 |
| apararti | OPF | 24.75% | 74.26% | 3.59% | 0.683 |
| opf-kz-ru | OPF | 25.74% | 70.30% | 3.74% | 0.646 |
| opf-ru-v2+sent300 | OPF | 26.73% | 63.37% | 2.17% | 0.690 |
| opf-ru-v2+ov100 | OPF | 27.72% | 62.38% | 2.75% | 0.664 |
| traciora | OPF | 28.71% | 63.37% | 3.16% | 0.695 |
| opf-ru-v2 | OPF | 28.71% | 61.39% | 2.23% | 0.680 |
| nuner-zero | GLINER | 29.70% | 64.36% | 5.14% | 0.446 |
| openai-base | OPF | 36.63% | 56.44% | 1.15% | 0.671 |
| rules-ru | RULES | 36.63% | 61.39% | 1.34% | 0.658 |
| openmed-multilingual | HF | 37.62% | 55.45% | 5.07% | 0.419 |
| gliner-stream-pii | GLINER | 38.61% | 54.46% | 2.75% | 0.546 |
| gliner-nvidia+sent300 | GLINER | 38.61% | 56.44% | 3.70% | 0.554 |
| gliner-nvidia | GLINER | 39.60% | 56.44% | 2.29% | 0.602 |
| fef2-secret-ru | HF | 40.59% | 51.49% | 0.37% | 0.597 |
| gliner-nvidia+ov100 | GLINER | 40.59% | 55.45% | 2.33% | 0.595 |
| gliner25-fastino+sent300 | GLINER2 | 41.58% | 54.46% | 10.53% | 0.287 |
| gliner25-fastino+ov100 | GLINER2 | 44.55% | 53.47% | 8.23% | 0.343 |
| gliner25-fastino | GLINER2 | 45.54% | 52.48% | 8.11% | 0.339 |
| gliner25-fastino+nochunk | GLINER2 | 46.53% | 51.49% | 5.51% | 0.450 |
| gliner-urchade | GLINER | 50.50% | 45.54% | 9.10% | 0.283 |
| ru-legal-ner | HF | 50.50% | 26.73% | 5.36% | 0.276 |
| nym-small | ONNX | 51.49% | 38.61% | 4.00% | 0.401 |
| nym-base+sent300 | HF | 51.49% | 35.64% | 3.78% | 0.357 |
| ru-legal-ner+ov100 | HF | 51.49% | 25.74% | 5.86% | 0.264 |
| ru-legal-ner+sent300 | HF | 51.49% | 24.75% | 7.87% | 0.223 |
| nym-base | HF | 53.47% | 34.65% | 3.29% | 0.370 |
| kalyan-ettin | HF | 53.47% | 38.61% | 3.81% | 0.364 |
| nym-base+ov100 | HF | 53.47% | 34.65% | 3.16% | 0.372 |
| gliner2-hivetrace-omni | GLINER2 | 55.45% | 43.56% | 2.83% | 0.413 |
| gliner2-large | GLINER2 | 55.45% | 40.59% | 6.04% | 0.256 |
| mmbert32k | HF | 57.43% | 22.77% | 2.71% | 0.329 |
| mmbert32k+nochunk | HF | 58.42% | 21.78% | 2.51% | 0.316 |
| openmed-nemotron | HF | 60.40% | 30.69% | 4.25% | 0.325 |
| bardsai-eu | ONNX | 61.39% | 26.73% | 4.46% | 0.317 |
| gravitee-small | HF | 63.37% | 27.72% | 1.27% | 0.324 |
| gliner-multi-v21 | GLINER | 71.29% | 22.77% | 3.67% | 0.195 |
| stanza-ru | STANZA | 71.29% | 14.85% | 20.40% | 0.114 |
| gliner2-hivetrace-uni | GLINER2 | 81.19% | 18.81% | 0.54% | 0.267 |
| ru-pii-ner | RUPII | 83.17% | 11.88% | 0.76% | 0.254 |
| gliner-pii-base | GLINER | 83.17% | 16.83% | 0.96% | 0.268 |
| gliner2-vladlinv | GLINER2 | 89.11% | 8.91% | 0.00% | 0.093 |
| ner-ru-yqelz | HF | 92.08% | 5.94% | 8.55% | 0.034 |
| spacy-ru-lg | SPACY | 95.05% | 4.95% | 3.43% | 0.061 |
| natasha | NATASHA | 96.04% | 3.96% | 0.55% | 0.079 |
| davlan-mbert | HF | 99.01% | 0.99% | 0.79% | 0.011 |
| ner-ru-gherman | HF | 100.00% | 0.00% | 0.09% | 0.000 |
| davlan-xlmr | HF | 100.00% | 0.00% | 0.88% | 0.000 |
| spacy-alrosait | SPACY | 100.00% | 0.00% | 0.00% | 0.000 |