EN · SECRETS
secrets-issues PII detection benchmark results
A frozen evaluation slice with 500 rows, 288 normalized gold annotations and source-level provenance.
500input rows
288gold annotations
1,168,779characters
82measured detectors
Dataset provenance
- Language group
- EN
- Task
- SECRETS
- License
- cc-by-4.0
- Data categories
- SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pii-shield-onnx | ONNX | 83.68% | 99.65% | 38.24% | 0.124 |
| pplx+cpu-int8 | PPLX | 81.25% | 100.00% | 18.81% | 0.208 |
| opf-kz-ru | OPF | 76.39% | 96.88% | 7.23% | 0.327 |
| pplx+ov100 | PPLX | 75.69% | 96.18% | 9.77% | 0.303 |
| pplx | PPLX | 75.00% | 95.83% | 9.88% | 0.288 |
| apararti | OPF | 74.65% | 97.92% | 6.67% | 0.344 |
| pplx+sent300 | PPLX | 73.61% | 96.88% | 10.68% | 0.289 |
| opf-ru | OPF | 72.92% | 98.26% | 5.85% | 0.372 |
| bardsai-eu | ONNX | 72.57% | 95.49% | 5.97% | 0.361 |
| traciora | OPF | 71.88% | 96.88% | 3.73% | 0.416 |
| opf-ru-v2+sent300 | OPF | 71.88% | 96.53% | 3.78% | 0.411 |
| nym-base+sent300 | HF | 71.53% | 98.96% | 8.05% | 0.327 |
| opf-ru-v2 | OPF | 71.18% | 95.14% | 3.53% | 0.418 |
| openai-base | OPF | 71.18% | 92.71% | 4.09% | 0.402 |
| opf-ru-v2+ov100 | OPF | 71.18% | 94.79% | 3.57% | 0.409 |
| openai-base-onnx | ONNX | 70.83% | 92.71% | 4.09% | 0.402 |
| nym-base+ov100 | HF | 70.49% | 99.31% | 6.65% | 0.350 |
| nym-base | HF | 69.44% | 98.61% | 6.72% | 0.350 |
| nym-base+cpu-int8 | HF | 69.44% | 98.26% | 7.19% | 0.343 |
| openmed-nemotron | HF | 68.40% | 94.10% | 11.76% | 0.269 |
| nym-small | ONNX | 67.01% | 97.92% | 7.56% | 0.331 |
| openmed-multilingual | HF | 65.97% | 93.06% | 10.31% | 0.284 |
| gliner-pii-edge | GLINER | 64.93% | 87.15% | 7.94% | 0.352 |
| kalyan-ettin | HF | 60.42% | 88.19% | 6.59% | 0.344 |
| credsweeper-noml | LEAKS | 60.07% | 81.94% | 1.82% | 0.477 |
| ru-legal-ner+ov100 | HF | 56.60% | 86.81% | 10.23% | 0.132 |
| mmbert32k+cpu-int8 | HF | 55.90% | 94.44% | 9.56% | 0.300 |
| credsweeper | LEAKS | 55.90% | 73.61% | 1.35% | 0.490 |
| ru-legal-ner+sent300 | HF | 54.17% | 82.64% | 10.45% | 0.125 |
| ru-legal-ner | HF | 53.47% | 84.38% | 9.27% | 0.129 |
| ru-legal-ner+cpu-int8 | HF | 53.47% | 84.03% | 9.44% | 0.128 |
| kalyan-ettin+cpu-int8 | HF | 53.13% | 78.82% | 3.75% | 0.416 |
| nuner-zero | GLINER | 49.31% | 70.49% | 2.53% | 0.308 |
| rules-ru | RULES | 48.96% | 67.71% | 10.19% | 0.246 |
| mmbert32k+nochunk | HF | 46.53% | 83.68% | 7.24% | 0.337 |
| gliner-nvidia+ov100 | GLINER | 45.83% | 64.93% | 3.38% | 0.463 |
| gitleaks | LEAKS | 45.49% | 54.86% | 0.81% | 0.538 |
| betterleaks | LEAKS | 44.79% | 57.29% | 1.00% | 0.541 |
| mmbert32k | HF | 44.44% | 81.60% | 9.77% | 0.298 |
| gliner-urchade | GLINER | 43.75% | 61.81% | 2.72% | 0.288 |
| gliner2-fastino | GLINER2 | 42.71% | 65.28% | 5.70% | 0.352 |
| gliner-nvidia+sent300 | GLINER | 42.36% | 60.42% | 4.45% | 0.401 |
| gliner-stream-pii | GLINER | 40.97% | 57.99% | 2.77% | 0.501 |
| gliner-nvidia | GLINER | 40.63% | 57.99% | 3.19% | 0.449 |
| gliner25-fastino+ov100 | GLINER2 | 36.46% | 54.17% | 5.58% | 0.128 |
| deepsecrets | LEAKS | 34.72% | 44.44% | 0.68% | 0.228 |
| fef2-secret-ru | HF | 32.99% | 51.39% | 0.44% | 0.544 |
| fef2-secret-ru+cpu-int8 | HF | 30.90% | 49.65% | 0.25% | 0.563 |
| gliner25-fastino+sent300 | GLINER2 | 27.43% | 46.18% | 7.31% | 0.188 |
| ner-ru-yqelz | HF | 25.35% | 34.38% | 8.70% | 0.107 |
| gliner25-fastino | GLINER2 | 24.31% | 40.97% | 5.40% | 0.140 |
| detect-secrets | LEAKS | 24.31% | 26.04% | 1.82% | 0.371 |
| titus | LEAKS | 24.31% | 29.86% | 0.89% | 0.420 |
| noseyparker | LEAKS | 23.26% | 30.21% | 0.83% | 0.421 |
| gliner2-hivetrace-omni | GLINER2 | 20.83% | 35.07% | 1.27% | 0.284 |
| stanza-ru | STANZA | 19.79% | 37.85% | 17.39% | 0.179 |
| gliner25-fastino+nochunk | GLINER2 | 19.10% | 34.38% | 2.79% | 0.103 |
| kingfisher | LEAKS | 13.89% | 15.97% | 0.84% | 0.441 |
| gliner2-large | GLINER2 | 13.54% | 32.29% | 2.31% | 0.090 |
| trufflehog | LEAKS | 12.50% | 15.28% | 0.38% | 0.164 |
| gravitee-small | HF | 12.15% | 25.69% | 3.64% | 0.077 |
| gliner-multi-v21 | GLINER | 11.81% | 29.51% | 2.07% | 0.095 |
| gravitee-small+cpu-int8 | HF | 11.46% | 24.65% | 3.23% | 0.079 |
| gliner-pii-edge+cpu-int8 | GLINER | 11.11% | 15.63% | 0.55% | 0.262 |
| gliner-pii-base | GLINER | 9.38% | 14.58% | 0.72% | 0.168 |
| gliner2-hivetrace-uni | GLINER2 | 7.99% | 13.89% | 0.45% | 0.074 |
| ru-pii-ner | RUPII | 7.29% | 12.15% | 1.26% | 0.242 |
| gliner2-vladlinv | GLINER2 | 6.60% | 9.72% | 0.04% | 0.158 |
| spacy-ru-lg | SPACY | 3.47% | 4.86% | 2.51% | 0.439 |
| natasha | NATASHA | 0.69% | 1.04% | 0.62% | 0.193 |
| ner-ru-gherman | HF | 0.00% | 0.00% | 0.11% | 0.000 |
| davlan-xlmr | HF | 0.00% | 0.35% | 0.49% | 0.000 |
| davlan-mbert | HF | 0.00% | 1.04% | 0.70% | 0.002 |
| spacy-alrosait | SPACY | 0.00% | 0.00% | 0.00% | 0.000 |
| davlan-mbert+cpu-int8 | HF | 0.00% | 1.39% | 0.53% | 0.002 |
| ner-ru-gherman-onnx | ONNX | 0.00% | 0.00% | 0.11% | 0.000 |
| ner-ru-gherman+cpu-int8 | HF | 0.00% | 0.00% | 0.10% | 0.000 |
| davlan-xlmr+cpu-int8 | HF | 0.00% | 0.35% | 0.11% | 0.001 |
| gliner-pii-base+cpu-int8 | GLINER | 0.00% | 0.00% | 0.00% | 0.000 |
| gliner-multi-v21+cpu-int8 | GLINER | 0.00% | 0.00% | 0.00% | 0.000 |
| gliner-nvidia+cpu-int8 | GLINER | 0.00% | 0.00% | 0.00% | 0.000 |
| gliner-urchade+cpu-int8 | GLINER | 0.00% | 0.00% | 0.00% | 0.000 |