MULTI · PII
nym-multi PII detection benchmark results
A frozen evaluation slice with 1,500 rows, 8,169 normalized gold annotations and source-level provenance.
1,500input rows
8,169gold annotations
391,219characters
39measured detectors
Dataset provenance
- Language group
- MULTI
- Task
- PII
- License
- mit
- Data categories
- ACCOUNT, ADDRESS, CONTACT, DATE, ID, NET, ORG, OTHER, PERSON, SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by untouched annotations. Training-source overlaps are excluded.
| Detector | Family | Untouched ↓ | Fully hidden ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pplx | PPLX | 2.78% | 96.22% | 5.43% | 0.933 |
| mmbert32k | HF | 6.99% | 79.09% | 5.81% | 0.857 |
| pii-shield-onnx | ONNX | 12.98% | 79.21% | 1.87% | 0.455 |
| openmed-multilingual | HF | 15.22% | 77.39% | 6.93% | 0.869 |
| bardsai-eu | ONNX | 15.57% | 76.80% | 0.94% | 0.864 |
| apararti | OPF | 19.66% | 76.02% | 3.93% | 0.859 |
| opf-ru | OPF | 19.99% | 68.10% | 5.24% | 0.816 |
| gliner-nvidia | GLINER | 20.44% | 77.50% | 8.05% | 0.831 |
| opf-kz-ru | OPF | 21.26% | 74.17% | 3.18% | 0.830 |
| gliner2-fastino | GLINER2 | 24.57% | 73.14% | 1.12% | 0.826 |
| openai-base | OPF | 25.12% | 70.94% | 3.18% | 0.823 |
| ru-legal-ner | HF | 26.12% | 58.01% | 12.73% | 0.461 |
| gliner2-hivetrace-omni | GLINER2 | 26.66% | 70.19% | 0.00% | 0.805 |
| openmed-nemotron | HF | 28.35% | 61.05% | 4.12% | 0.686 |
| kalyan-ettin | HF | 29.64% | 59.48% | 2.62% | 0.772 |
| gravitee-small | HF | 33.37% | 62.16% | 4.12% | 0.604 |
| nuner-zero | GLINER | 36.48% | 59.29% | 4.31% | 0.734 |
| gliner-pii-edge | GLINER | 36.76% | 60.29% | 13.86% | 0.714 |
| ru-pii-ner | RUPII | 37.68% | 60.29% | 3.56% | 0.751 |
| opf-ru-v2 | OPF | 39.27% | 52.59% | 0.00% | 0.736 |
| traciora | OPF | 42.22% | 49.27% | 0.00% | 0.692 |
| gliner-stream-pii | GLINER | 42.50% | 51.65% | 7.30% | 0.658 |
| gliner2-hivetrace-uni | GLINER2 | 43.71% | 53.34% | 1.12% | 0.709 |
| gliner2-large | GLINER2 | 44.66% | 52.85% | 0.00% | 0.681 |
| gliner-urchade | GLINER | 45.60% | 53.32% | 0.00% | 0.721 |
| gliner2-vladlinv | GLINER2 | 45.91% | 53.63% | 0.00% | 0.681 |
| gliner25-fastino | GLINER2 | 58.55% | 39.58% | 0.00% | 0.578 |
| gliner-pii-base | GLINER | 59.66% | 39.11% | 0.75% | 0.573 |
| gliner-multi-v21 | GLINER | 60.01% | 37.42% | 0.00% | 0.475 |
| davlan-xlmr | HF | 62.70% | 33.10% | 0.94% | 0.423 |
| ner-ru-yqelz | HF | 62.76% | 33.10% | 0.00% | 0.401 |
| davlan-mbert | HF | 64.24% | 31.02% | 0.94% | 0.401 |
| ner-ru-gherman | HF | 66.86% | 27.14% | 0.00% | 0.351 |
| stanza-ru | STANZA | 78.87% | 16.55% | 11.24% | 0.211 |
| fef2-secret-ru | HF | 86.64% | 11.74% | 0.00% | 0.280 |
| rules-ru | RULES | 89.58% | 9.95% | 0.00% | 0.311 |
| spacy-ru-lg | SPACY | 90.35% | 8.67% | 0.00% | 0.125 |
| natasha | NATASHA | 96.68% | 2.83% | 0.00% | 0.046 |
| spacy-alrosait | SPACY | 98.27% | 1.44% | 0.00% | 0.027 |