RU · SECRETS
synth-secrets-ru PII detection benchmark results
A frozen evaluation slice with 600 rows, 581 normalized gold annotations and source-level provenance.
600input rows
581gold annotations
274,241characters
49measured detectors
Dataset provenance
- Language group
- RU
- Task
- SECRETS
- License
- cc-by-4.0
- Data categories
- SECRET
- Project generated
- Yes
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pplx | PPLX | 97.59% | 100.00% | 44.69% | 0.328 |
| pii-shield-onnx | ONNX | 97.42% | 99.66% | 70.31% | 0.241 |
| opf-kz-ru | OPF | 94.15% | 99.31% | 49.04% | 0.314 |
| apararti | OPF | 91.74% | 99.31% | 52.15% | 0.300 |
| openai-base | OPF | 91.74% | 97.93% | 51.56% | 0.304 |
| rules-ru | RULES | 81.41% | 81.93% | 16.74% | 0.518 |
| opf-ru-v2 | OPF | 80.90% | 97.42% | 33.15% | 0.381 |
| opf-ru | OPF | 79.69% | 99.83% | 31.14% | 0.395 |
| traciora | OPF | 77.28% | 97.07% | 32.45% | 0.392 |
| openmed-multilingual | HF | 72.12% | 96.56% | 39.35% | 0.334 |
| gliner-pii-edge | GLINER | 71.26% | 85.20% | 26.68% | 0.391 |
| bardsai-eu | ONNX | 70.22% | 90.53% | 28.24% | 0.400 |
| nym-base | HF | 68.16% | 96.39% | 34.32% | 0.346 |
| gliner-nvidia | GLINER | 61.62% | 63.86% | 12.01% | 0.461 |
| nym-small | ONNX | 60.93% | 94.49% | 32.58% | 0.350 |
| gliner2-fastino | GLINER2 | 59.55% | 64.89% | 4.45% | 0.475 |
| gliner25-fastino | GLINER2 | 56.80% | 60.07% | 24.40% | 0.266 |
| fef2-secret-ru | HF | 56.11% | 74.18% | 5.22% | 0.571 |
| openmed-nemotron | HF | 53.53% | 85.20% | 34.25% | 0.345 |
| gliner2-hivetrace-omni | GLINER2 | 53.01% | 53.36% | 7.62% | 0.400 |
| nuner-zero | GLINER | 52.15% | 65.92% | 5.14% | 0.491 |
| gliner25-fastino-ru | GLINER2 | 51.98% | 54.39% | 14.51% | 0.335 |
| gliner2-fastino-ru | GLINER2 | 46.47% | 51.12% | 2.82% | 0.394 |
| kalyan-ettin | HF | 44.23% | 85.20% | 26.93% | 0.381 |
| gliner-nvidia-ru | GLINER | 42.34% | 43.03% | 3.52% | 0.424 |
| gliner2-hivetrace-omni-ru | GLINER2 | 41.65% | 41.82% | 8.13% | 0.290 |
| gliner-stream-pii | GLINER | 40.62% | 47.50% | 4.26% | 0.370 |
| ru-legal-ner | HF | 35.63% | 91.05% | 29.02% | 0.369 |
| gliner2-vladlinv-ru | GLINER2 | 33.22% | 35.28% | 8.46% | 0.203 |
| gliner-urchade | GLINER | 32.01% | 34.94% | 1.63% | 0.307 |
| stanza-ru | STANZA | 28.40% | 59.04% | 14.83% | 0.303 |
| gliner2-vladlinv | GLINER2 | 28.06% | 30.64% | 3.43% | 0.211 |
| gliner2-large | GLINER2 | 23.75% | 26.68% | 3.45% | 0.216 |
| gravitee-small | HF | 23.41% | 42.51% | 10.98% | 0.187 |
| mmbert32k | HF | 22.03% | 76.59% | 33.94% | 0.256 |
| gliner-pii-base | GLINER | 21.51% | 23.41% | 1.91% | 0.213 |
| gliner-urchade-ru | GLINER | 20.48% | 21.34% | 0.91% | 0.193 |
| gliner-multi-v21-ru | GLINER | 16.18% | 22.03% | 1.50% | 0.123 |
| gliner-multi-v21 | GLINER | 14.46% | 19.97% | 1.57% | 0.103 |
| ru-pii-ner | RUPII | 12.91% | 17.90% | 5.05% | 0.136 |
| gliner2-hivetrace-uni-ru | GLINER2 | 11.70% | 15.49% | 3.05% | 0.136 |
| ner-ru-yqelz | HF | 5.16% | 9.29% | 13.91% | 0.076 |
| gliner2-hivetrace-uni | GLINER2 | 1.03% | 1.03% | 0.21% | 0.010 |
| spacy-ru-lg | SPACY | 0.69% | 0.69% | 0.27% | 0.008 |
| natasha | NATASHA | 0.17% | 0.17% | 0.63% | 0.002 |
| ner-ru-gherman | HF | 0.00% | 0.00% | 0.00% | 0.000 |
| davlan-xlmr | HF | 0.00% | 0.00% | 0.00% | 0.000 |
| davlan-mbert | HF | 0.00% | 0.00% | 0.05% | 0.000 |
| spacy-alrosait | SPACY | 0.00% | 0.00% | 0.00% | 0.000 |