MULTI · PII
nym-multi PII detection benchmark results
A frozen evaluation slice with 1,500 rows, 8,169 normalized gold annotations and source-level provenance.
1,500input rows
8,169gold annotations
391,219characters
39measured detectors
Dataset provenance
- Language group
- MULTI
- Task
- PII
- License
- mit
- Data categories
- ACCOUNT, ADDRESS, CONTACT, DATE, ID, NET, ORG, OTHER, PERSON, SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pplx | PPLX | 96.22% | 97.22% | 5.43% | 0.933 |
| pii-shield-onnx | ONNX | 79.21% | 87.02% | 1.87% | 0.455 |
| mmbert32k | HF | 79.09% | 93.01% | 5.81% | 0.857 |
| gliner-nvidia | GLINER | 77.50% | 79.56% | 8.05% | 0.831 |
| openmed-multilingual | HF | 77.39% | 84.78% | 6.93% | 0.869 |
| bardsai-eu | ONNX | 76.80% | 84.43% | 0.94% | 0.864 |
| apararti | OPF | 76.02% | 80.34% | 3.93% | 0.859 |
| opf-kz-ru | OPF | 74.17% | 78.74% | 3.18% | 0.830 |
| gliner2-fastino | GLINER2 | 73.14% | 75.43% | 1.12% | 0.826 |
| openai-base | OPF | 70.94% | 74.88% | 3.18% | 0.823 |
| gliner2-hivetrace-omni | GLINER2 | 70.19% | 73.34% | 0.00% | 0.805 |
| opf-ru | OPF | 68.10% | 80.01% | 5.24% | 0.816 |
| gravitee-small | HF | 62.16% | 66.63% | 4.12% | 0.604 |
| openmed-nemotron | HF | 61.05% | 71.65% | 4.12% | 0.686 |
| ru-pii-ner | RUPII | 60.29% | 62.32% | 3.56% | 0.751 |
| gliner-pii-edge | GLINER | 60.29% | 63.24% | 13.86% | 0.714 |
| kalyan-ettin | HF | 59.48% | 70.36% | 2.62% | 0.772 |
| nuner-zero | GLINER | 59.29% | 63.52% | 4.31% | 0.734 |
| ru-legal-ner | HF | 58.01% | 73.88% | 12.73% | 0.461 |
| gliner2-vladlinv | GLINER2 | 53.63% | 54.09% | 0.00% | 0.681 |
| gliner2-hivetrace-uni | GLINER2 | 53.34% | 56.29% | 1.12% | 0.709 |
| gliner-urchade | GLINER | 53.32% | 54.40% | 0.00% | 0.721 |
| gliner2-large | GLINER2 | 52.85% | 55.34% | 0.00% | 0.681 |
| opf-ru-v2 | OPF | 52.59% | 60.73% | 0.00% | 0.736 |
| gliner-stream-pii | GLINER | 51.65% | 57.50% | 7.30% | 0.658 |
| traciora | OPF | 49.27% | 57.78% | 0.00% | 0.692 |
| gliner25-fastino | GLINER2 | 39.58% | 41.45% | 0.00% | 0.578 |
| gliner-pii-base | GLINER | 39.11% | 40.34% | 0.75% | 0.573 |
| gliner-multi-v21 | GLINER | 37.42% | 39.99% | 0.00% | 0.475 |
| ner-ru-yqelz | HF | 33.10% | 37.24% | 0.00% | 0.401 |
| davlan-xlmr | HF | 33.10% | 37.30% | 0.94% | 0.423 |
| davlan-mbert | HF | 31.02% | 35.76% | 0.94% | 0.401 |
| ner-ru-gherman | HF | 27.14% | 33.14% | 0.00% | 0.351 |
| stanza-ru | STANZA | 16.55% | 21.13% | 11.24% | 0.211 |
| fef2-secret-ru | HF | 11.74% | 13.36% | 0.00% | 0.280 |
| rules-ru | RULES | 9.95% | 10.42% | 0.00% | 0.311 |
| spacy-ru-lg | SPACY | 8.67% | 9.65% | 0.00% | 0.125 |
| natasha | NATASHA | 2.83% | 3.32% | 0.00% | 0.046 |
| spacy-alrosait | SPACY | 1.44% | 1.73% | 0.00% | 0.027 |