RU · PII
jayguard PII detection benchmark results
A frozen evaluation slice with 850 rows, 1,195 normalized gold annotations and source-level provenance.
850input rows
1,195gold annotations
427,476characters
66measured detectors
Dataset provenance
- Language group
- RU
- Task
- PII
- License
- mit (yaml) / apache-2.0 (card text)
- Data categories
- ADDRESS, PERSON
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| gliner2-fastino | GLINER2 | 77.57% | 89.46% | 2.51% | 0.585 |
| gliner2-fastino-ru | GLINER2 | 76.90% | 88.37% | 1.27% | 0.666 |
| gliner-urchade | GLINER | 76.40% | 86.86% | 2.37% | 0.567 |
| gliner25-fastino+sent300 | GLINER2 | 74.98% | 87.11% | 2.46% | 0.560 |
| gliner25-fastino+ov100 | GLINER2 | 74.73% | 86.61% | 2.03% | 0.589 |
| gliner25-fastino+nochunk | GLINER2 | 74.64% | 86.78% | 1.61% | 0.615 |
| gliner25-fastino | GLINER2 | 74.48% | 86.44% | 1.85% | 0.597 |
| gliner2-hivetrace-omni | GLINER2 | 73.72% | 86.19% | 2.38% | 0.588 |
| gliner-multi-v21 | GLINER | 73.22% | 86.78% | 2.48% | 0.556 |
| gliner25-fastino-ru | GLINER2 | 73.14% | 85.77% | 1.29% | 0.623 |
| pplx+sent300 | PPLX | 72.97% | 84.69% | 2.37% | 0.552 |
| gliner25-fastino-ru+nochunk | GLINER2 | 72.97% | 85.69% | 1.35% | 0.632 |
| pplx+ov100 | PPLX | 72.30% | 83.93% | 2.16% | 0.553 |
| pplx | PPLX | 71.97% | 83.51% | 2.11% | 0.555 |
| gliner2-hivetrace-omni-ru | GLINER2 | 71.21% | 84.77% | 1.89% | 0.601 |
| nuner-zero | GLINER | 71.05% | 84.69% | 1.71% | 0.608 |
| gliner-multi-v21-ru | GLINER | 69.29% | 82.68% | 1.39% | 0.613 |
| bardsai-eu | ONNX | 68.95% | 82.85% | 2.20% | 0.525 |
| ru-pii-ner | RUPII | 67.78% | 79.08% | 1.45% | 0.556 |
| davlan-xlmr | HF | 67.20% | 82.93% | 1.24% | 0.617 |
| gliner2-vladlinv-ru | GLINER2 | 64.60% | 74.81% | 0.68% | 0.635 |
| gliner2-vladlinv | GLINER2 | 64.52% | 74.81% | 0.71% | 0.630 |
| gliner-nvidia+sent300 | GLINER | 62.68% | 77.07% | 1.98% | 0.574 |
| gliner-nvidia+ov100 | GLINER | 61.76% | 75.82% | 1.56% | 0.602 |
| ru-legal-ner+sent300 | HF | 61.67% | 80.25% | 4.64% | 0.445 |
| gliner-nvidia | GLINER | 61.09% | 75.40% | 1.39% | 0.611 |
| ner-ru-yqelz | HF | 60.33% | 82.59% | 6.22% | 0.443 |
| ru-legal-ner+ov100 | HF | 59.41% | 78.33% | 4.17% | 0.461 |
| ru-legal-ner | HF | 59.00% | 77.82% | 3.99% | 0.467 |
| gliner2-hivetrace-uni | GLINER2 | 58.74% | 71.46% | 1.68% | 0.579 |
| gliner2-large | GLINER2 | 58.58% | 73.39% | 2.44% | 0.512 |
| nym-base+ov100 | HF | 56.40% | 86.86% | 2.69% | 0.544 |
| nym-base | HF | 56.32% | 86.95% | 2.62% | 0.546 |
| nym-base+sent300 | HF | 56.32% | 86.78% | 2.75% | 0.540 |
| gliner-nvidia-ru | GLINER | 56.15% | 71.13% | 0.99% | 0.616 |
| davlan-mbert | HF | 55.23% | 75.40% | 1.37% | 0.572 |
| fef2-secret-ru | HF | 54.48% | 79.67% | 1.22% | 0.563 |
| traciora | OPF | 53.72% | 66.44% | 1.85% | 0.467 |
| spacy-ru-lg | SPACY | 52.55% | 79.25% | 3.26% | 0.488 |
| stanza-ru | STANZA | 51.55% | 73.97% | 8.92% | 0.299 |
| opf-ru | OPF | 49.04% | 73.22% | 3.32% | 0.460 |
| mmbert32k | HF | 48.79% | 79.41% | 3.12% | 0.486 |
| nym-small | ONNX | 48.79% | 78.49% | 2.66% | 0.508 |
| apararti | OPF | 48.54% | 61.67% | 3.09% | 0.423 |
| spacy-alrosait | SPACY | 48.20% | 59.75% | 1.43% | 0.537 |
| pii-shield-onnx | ONNX | 47.70% | 68.28% | 6.02% | 0.408 |
| gliner-pii-edge | GLINER | 46.95% | 71.21% | 2.00% | 0.495 |
| openai-base | OPF | 46.03% | 54.98% | 1.85% | 0.445 |
| mmbert32k+nochunk | HF | 45.86% | 75.90% | 3.01% | 0.480 |
| opf-ru-v2+sent300 | OPF | 45.61% | 58.74% | 1.25% | 0.487 |
| opf-ru-v2+ov100 | OPF | 45.52% | 58.49% | 1.20% | 0.492 |
| gliner-stream-pii | GLINER | 45.02% | 61.51% | 1.09% | 0.526 |
| opf-ru-v2 | OPF | 44.60% | 57.32% | 1.15% | 0.492 |
| natasha | NATASHA | 44.35% | 61.09% | 2.36% | 0.420 |
| gliner-pii-base | GLINER | 43.68% | 63.18% | 1.60% | 0.474 |
| openmed-nemotron | HF | 38.83% | 64.85% | 4.32% | 0.368 |
| ner-ru-gherman | HF | 37.15% | 79.00% | 0.88% | 0.595 |
| gliner-urchade-ru | GLINER | 33.56% | 34.39% | 0.43% | 0.514 |
| kalyan-ettin | HF | 29.21% | 52.47% | 2.13% | 0.365 |
| openmed-multilingual | HF | 28.70% | 58.74% | 2.90% | 0.404 |
| opf-kz-ru | OPF | 26.69% | 44.35% | 2.35% | 0.361 |
| gravitee-small | HF | 22.01% | 29.62% | 2.95% | 0.255 |
| gliner2-hivetrace-uni-ru | GLINER2 | 18.08% | 24.77% | 0.94% | 0.276 |
| rules-ru | RULES | 0.08% | 0.08% | 0.84% | 0.000 |
| betterleaks | LEAKS | 0.00% | 0.00% | 0.00% | 0.000 |
| gitleaks | LEAKS | 0.00% | 0.00% | 0.00% | 0.000 |