EN · SECRETS
secrets-issues PII detection benchmark results
A frozen evaluation slice with 500 rows, 288 normalized gold annotations and source-level provenance.
500input rows
288gold annotations
1,168,779characters
82measured detectors
Dataset provenance
- Language group
- EN
- Task
- SECRETS
- License
- cc-by-4.0
- Data categories
- SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by untouched annotations. Training-source overlaps are excluded.
| Detector | Family | Untouched ↓ | Fully hidden ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pplx+cpu-int8 | PPLX | 0.00% | 81.25% | 18.81% | 0.208 |
| pii-shield-onnx | ONNX | 0.35% | 83.68% | 38.24% | 0.124 |
| nym-base+ov100 | HF | 0.69% | 70.49% | 6.65% | 0.350 |
| nym-base+sent300 | HF | 1.04% | 71.53% | 8.05% | 0.327 |
| nym-base | HF | 1.39% | 69.44% | 6.72% | 0.350 |
| opf-ru | OPF | 1.74% | 72.92% | 5.85% | 0.372 |
| nym-base+cpu-int8 | HF | 1.74% | 69.44% | 7.19% | 0.343 |
| nym-small | ONNX | 2.08% | 67.01% | 7.56% | 0.331 |
| apararti | OPF | 2.08% | 74.65% | 6.67% | 0.344 |
| traciora | OPF | 3.13% | 71.88% | 3.73% | 0.416 |
| opf-kz-ru | OPF | 3.13% | 76.39% | 7.23% | 0.327 |
| pplx+sent300 | PPLX | 3.13% | 73.61% | 10.68% | 0.289 |
| opf-ru-v2+sent300 | OPF | 3.47% | 71.88% | 3.78% | 0.411 |
| pplx+ov100 | PPLX | 3.82% | 75.69% | 9.77% | 0.303 |
| pplx | PPLX | 4.17% | 75.00% | 9.88% | 0.288 |
| bardsai-eu | ONNX | 4.51% | 72.57% | 5.97% | 0.361 |
| opf-ru-v2 | OPF | 4.86% | 71.18% | 3.53% | 0.418 |
| opf-ru-v2+ov100 | OPF | 5.21% | 71.18% | 3.57% | 0.409 |
| mmbert32k+cpu-int8 | HF | 5.56% | 55.90% | 9.56% | 0.300 |
| openmed-nemotron | HF | 5.90% | 68.40% | 11.76% | 0.269 |
| openmed-multilingual | HF | 6.94% | 65.97% | 10.31% | 0.284 |
| openai-base | OPF | 7.29% | 71.18% | 4.09% | 0.402 |
| openai-base-onnx | ONNX | 7.29% | 70.83% | 4.09% | 0.402 |
| kalyan-ettin | HF | 11.81% | 60.42% | 6.59% | 0.344 |
| gliner-pii-edge | GLINER | 12.85% | 64.93% | 7.94% | 0.352 |
| ru-legal-ner+ov100 | HF | 13.19% | 56.60% | 10.23% | 0.132 |
| ru-legal-ner | HF | 15.63% | 53.47% | 9.27% | 0.129 |
| ru-legal-ner+cpu-int8 | HF | 15.97% | 53.47% | 9.44% | 0.128 |
| mmbert32k+nochunk | HF | 16.32% | 46.53% | 7.24% | 0.337 |
| ru-legal-ner+sent300 | HF | 17.36% | 54.17% | 10.45% | 0.125 |
| credsweeper-noml | LEAKS | 18.06% | 60.07% | 1.82% | 0.477 |
| mmbert32k | HF | 18.40% | 44.44% | 9.77% | 0.298 |
| kalyan-ettin+cpu-int8 | HF | 21.18% | 53.13% | 3.75% | 0.416 |
| credsweeper | LEAKS | 26.39% | 55.90% | 1.35% | 0.490 |
| nuner-zero | GLINER | 29.51% | 49.31% | 2.53% | 0.308 |
| rules-ru | RULES | 32.29% | 48.96% | 10.19% | 0.246 |
| gliner2-fastino | GLINER2 | 34.72% | 42.71% | 5.70% | 0.352 |
| gliner-nvidia+ov100 | GLINER | 35.07% | 45.83% | 3.38% | 0.463 |
| gliner-urchade | GLINER | 38.19% | 43.75% | 2.72% | 0.288 |
| gliner-nvidia+sent300 | GLINER | 39.58% | 42.36% | 4.45% | 0.401 |
| gliner-nvidia | GLINER | 42.01% | 40.63% | 3.19% | 0.449 |
| gliner-stream-pii | GLINER | 42.01% | 40.97% | 2.77% | 0.501 |
| betterleaks | LEAKS | 42.71% | 44.79% | 1.00% | 0.541 |
| gitleaks | LEAKS | 45.14% | 45.49% | 0.81% | 0.538 |
| gliner25-fastino+ov100 | GLINER2 | 45.83% | 36.46% | 5.58% | 0.128 |
| fef2-secret-ru | HF | 48.61% | 32.99% | 0.44% | 0.544 |
| fef2-secret-ru+cpu-int8 | HF | 50.35% | 30.90% | 0.25% | 0.563 |
| gliner25-fastino+sent300 | GLINER2 | 53.82% | 27.43% | 7.31% | 0.188 |
| deepsecrets | LEAKS | 55.56% | 34.72% | 0.68% | 0.228 |
| gliner25-fastino | GLINER2 | 59.03% | 24.31% | 5.40% | 0.140 |
| stanza-ru | STANZA | 62.15% | 19.79% | 17.39% | 0.179 |
| gliner2-hivetrace-omni | GLINER2 | 64.93% | 20.83% | 1.27% | 0.284 |
| ner-ru-yqelz | HF | 65.63% | 25.35% | 8.70% | 0.107 |
| gliner25-fastino+nochunk | GLINER2 | 65.63% | 19.10% | 2.79% | 0.103 |
| gliner2-large | GLINER2 | 67.71% | 13.54% | 2.31% | 0.090 |
| noseyparker | LEAKS | 69.79% | 23.26% | 0.83% | 0.421 |
| titus | LEAKS | 70.14% | 24.31% | 0.89% | 0.420 |
| gliner-multi-v21 | GLINER | 70.49% | 11.81% | 2.07% | 0.095 |
| detect-secrets | LEAKS | 73.96% | 24.31% | 1.82% | 0.371 |
| gravitee-small | HF | 74.31% | 12.15% | 3.64% | 0.077 |
| gravitee-small+cpu-int8 | HF | 75.35% | 11.46% | 3.23% | 0.079 |
| kingfisher | LEAKS | 84.03% | 13.89% | 0.84% | 0.441 |
| gliner-pii-edge+cpu-int8 | GLINER | 84.38% | 11.11% | 0.55% | 0.262 |
| trufflehog | LEAKS | 84.72% | 12.50% | 0.38% | 0.164 |
| gliner-pii-base | GLINER | 85.42% | 9.38% | 0.72% | 0.168 |
| gliner2-hivetrace-uni | GLINER2 | 86.11% | 7.99% | 0.45% | 0.074 |
| ru-pii-ner | RUPII | 87.85% | 7.29% | 1.26% | 0.242 |
| gliner2-vladlinv | GLINER2 | 90.28% | 6.60% | 0.04% | 0.158 |
| spacy-ru-lg | SPACY | 95.14% | 3.47% | 2.51% | 0.439 |
| davlan-mbert+cpu-int8 | HF | 98.61% | 0.00% | 0.53% | 0.002 |
| davlan-mbert | HF | 98.96% | 0.00% | 0.70% | 0.002 |
| natasha | NATASHA | 98.96% | 0.69% | 0.62% | 0.193 |
| davlan-xlmr | HF | 99.65% | 0.00% | 0.49% | 0.000 |
| davlan-xlmr+cpu-int8 | HF | 99.65% | 0.00% | 0.11% | 0.001 |
| ner-ru-gherman | HF | 100.00% | 0.00% | 0.11% | 0.000 |
| spacy-alrosait | SPACY | 100.00% | 0.00% | 0.00% | 0.000 |
| ner-ru-gherman-onnx | ONNX | 100.00% | 0.00% | 0.11% | 0.000 |
| ner-ru-gherman+cpu-int8 | HF | 100.00% | 0.00% | 0.10% | 0.000 |
| gliner-pii-base+cpu-int8 | GLINER | 100.00% | 0.00% | 0.00% | 0.000 |
| gliner-multi-v21+cpu-int8 | GLINER | 100.00% | 0.00% | 0.00% | 0.000 |
| gliner-nvidia+cpu-int8 | GLINER | 100.00% | 0.00% | 0.00% | 0.000 |
| gliner-urchade+cpu-int8 | GLINER | 100.00% | 0.00% | 0.00% | 0.000 |