EN · SECRETS
creddata PII detection benchmark results
A frozen evaluation slice with 1,477 rows, 776 normalized gold annotations and source-level provenance.
1,477input rows
776gold annotations
268,803characters
55measured detectors
Dataset provenance
- Language group
- EN
- Task
- SECRETS
- License
- apache-2.0 (meta); file texts under their repositories' licenses
- Data categories
- SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by untouched annotations. Training-source overlaps are excluded.
| Detector | Family | Untouched ↓ | Fully hidden ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| nym-base+sent300 | HF | 1.16% | 87.24% | 12.92% | 0.773 |
| nym-base | HF | 1.29% | 87.37% | 10.98% | 0.779 |
| nym-base+ov100 | HF | 1.29% | 87.37% | 10.91% | 0.781 |
| nym-small | ONNX | 1.93% | 83.76% | 14.27% | 0.778 |
| opf-kz-ru | OPF | 2.06% | 95.36% | 6.61% | 0.881 |
| traciora | OPF | 2.45% | 91.37% | 4.08% | 0.916 |
| apararti | OPF | 2.45% | 94.20% | 6.62% | 0.887 |
| opf-ru | OPF | 2.96% | 92.65% | 4.90% | 0.906 |
| pii-shield-onnx | ONNX | 3.74% | 92.14% | 40.24% | 0.585 |
| opf-ru-v2 | OPF | 4.38% | 90.85% | 2.56% | 0.932 |
| opf-ru-v2+sent300 | OPF | 4.38% | 90.59% | 2.69% | 0.924 |
| opf-ru-v2+ov100 | OPF | 4.51% | 90.98% | 2.65% | 0.933 |
| pplx | PPLX | 5.80% | 86.60% | 7.23% | 0.782 |
| pplx+ov100 | PPLX | 5.80% | 86.60% | 7.55% | 0.750 |
| pplx+sent300 | PPLX | 6.19% | 85.82% | 8.63% | 0.800 |
| gliner-pii-edge | GLINER | 7.86% | 80.03% | 11.53% | 0.616 |
| kalyan-ettin | HF | 7.99% | 77.84% | 4.71% | 0.786 |
| openmed-multilingual | HF | 8.76% | 75.77% | 10.54% | 0.775 |
| openai-base | OPF | 8.89% | 88.40% | 4.64% | 0.893 |
| ru-legal-ner | HF | 10.05% | 68.17% | 13.31% | 0.702 |
| ru-legal-ner+ov100 | HF | 10.05% | 68.17% | 14.02% | 0.700 |
| ru-legal-ner+sent300 | HF | 10.18% | 67.65% | 12.76% | 0.695 |
| mmbert32k | HF | 10.57% | 60.70% | 10.19% | 0.574 |
| mmbert32k+nochunk | HF | 10.95% | 60.44% | 8.89% | 0.555 |
| gliner25-fastino+sent300 | GLINER2 | 12.89% | 82.35% | 11.81% | 0.478 |
| gliner25-fastino+ov100 | GLINER2 | 13.40% | 82.09% | 10.91% | 0.444 |
| gliner25-fastino+nochunk | GLINER2 | 13.53% | 82.09% | 9.73% | 0.467 |
| gliner25-fastino | GLINER2 | 13.66% | 82.09% | 10.69% | 0.447 |
| gliner2-fastino | GLINER2 | 14.30% | 77.96% | 10.83% | 0.573 |
| openmed-nemotron | HF | 14.30% | 68.30% | 14.16% | 0.748 |
| gliner-stream-pii | GLINER | 14.56% | 77.84% | 6.89% | 0.540 |
| gliner-nvidia+sent300 | GLINER | 17.14% | 78.61% | 7.85% | 0.648 |
| gliner-nvidia+ov100 | GLINER | 17.78% | 78.61% | 6.95% | 0.641 |
| gliner-nvidia | GLINER | 18.04% | 78.35% | 6.94% | 0.603 |
| bardsai-eu | ONNX | 24.23% | 64.56% | 6.15% | 0.867 |
| nuner-zero | GLINER | 25.90% | 64.82% | 6.43% | 0.466 |
| fef2-secret-ru | HF | 34.41% | 63.40% | 2.63% | 0.516 |
| gliner-urchade | GLINER | 37.63% | 57.99% | 8.74% | 0.326 |
| gliner2-vladlinv | GLINER2 | 40.59% | 56.31% | 1.37% | 0.367 |
| ner-ru-yqelz | HF | 45.23% | 43.04% | 10.84% | 0.335 |
| gliner2-hivetrace-omni | GLINER2 | 48.07% | 49.61% | 2.85% | 0.351 |
| gravitee-small | HF | 50.52% | 38.92% | 3.31% | 0.371 |
| gliner2-large | GLINER2 | 52.19% | 44.07% | 6.76% | 0.217 |
| rules-ru | RULES | 60.31% | 38.02% | 12.01% | 0.643 |
| ru-pii-ner | RUPII | 61.08% | 34.54% | 2.17% | 0.301 |
| gliner-multi-v21 | GLINER | 75.52% | 20.36% | 4.52% | 0.073 |
| gliner2-hivetrace-uni | GLINER2 | 76.16% | 22.81% | 2.10% | 0.148 |
| stanza-ru | STANZA | 79.64% | 12.37% | 15.22% | 0.397 |
| gliner-pii-base | GLINER | 81.06% | 17.40% | 2.82% | 0.081 |
| spacy-ru-lg | SPACY | 89.18% | 10.31% | 4.22% | 0.174 |
| natasha | NATASHA | 99.48% | 0.00% | 0.69% | 0.006 |
| davlan-mbert | HF | 99.87% | 0.00% | 1.19% | 0.000 |
| ner-ru-gherman | HF | 100.00% | 0.00% | 0.06% | 0.000 |
| davlan-xlmr | HF | 100.00% | 0.00% | 0.83% | 0.000 |
| spacy-alrosait | SPACY | 100.00% | 0.00% | 0.00% | 0.000 |