MULTI · PII
redact-multi PII detection benchmark results
A frozen evaluation slice with 1,000 rows, 22,864 normalized gold annotations and source-level provenance.
1,000input rows
22,864gold annotations
1,962,299characters
41measured detectors
Dataset provenance
- Language group
- MULTI
- Task
- PII
- License
- other (gated, terms accepted)
- Data categories
- ACCOUNT, ADDRESS, CONTACT, DATE, ID, NET, ORG, OTHER, PERSON, SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pplx | PPLX | 86.98% | 89.39% | — | 0.705 |
| gliner-urchade | GLINER | 82.02% | 84.54% | — | 0.663 |
| bardsai-eu | ONNX | 79.98% | 91.65% | — | 0.755 |
| gliner2-fastino | GLINER2 | 78.92% | 84.54% | — | 0.694 |
| gliner2-hivetrace-omni | GLINER2 | 78.44% | 84.03% | — | 0.717 |
| nuner-zero | GLINER | 73.43% | 78.46% | — | 0.662 |
| gliner2-hivetrace-uni | GLINER2 | 72.38% | 76.73% | — | 0.689 |
| pii-shield-onnx | ONNX | 70.23% | 81.56% | — | 0.329 |
| apararti | OPF | 68.32% | 74.10% | — | 0.616 |
| openai-base | OPF | 68.17% | 72.01% | — | 0.623 |
| gliner25-fastino | GLINER2 | 67.60% | 71.45% | — | 0.675 |
| opf-kz-ru | OPF | 63.37% | 69.30% | — | 0.598 |
| mmbert32k | HF | 62.39% | 89.25% | — | 0.595 |
| gliner2-large | GLINER2 | 62.34% | 67.31% | — | 0.610 |
| gliner-nvidia | GLINER | 62.17% | 82.68% | — | 0.623 |
| gliner-multi-v21 | GLINER | 60.56% | 67.84% | — | 0.546 |
| nym-base | HF | 59.45% | 81.93% | — | 0.662 |
| gravitee-small | HF | 59.42% | 65.43% | — | 0.510 |
| opf-ru | OPF | 58.26% | 74.80% | — | 0.591 |
| nym-small | ONNX | 57.85% | 79.33% | — | 0.657 |
| ru-pii-ner | RUPII | 55.93% | 59.06% | — | 0.607 |
| davlan-xlmr | HF | 55.52% | 64.35% | — | 0.539 |
| openmed-multilingual | HF | 54.48% | 77.67% | — | 0.614 |
| davlan-mbert | HF | 53.74% | 61.69% | — | 0.507 |
| opf-ru-v2 | OPF | 52.35% | 61.83% | — | 0.647 |
| traciora | OPF | 52.29% | 60.69% | — | 0.650 |
| gliner2-vladlinv | GLINER2 | 49.27% | 50.86% | — | 0.571 |
| openmed-nemotron | HF | 46.51% | 71.09% | — | 0.484 |
| ner-ru-yqelz | HF | 45.79% | 55.84% | — | 0.460 |
| kalyan-ettin | HF | 44.22% | 68.89% | — | 0.543 |
| stanza-ru | STANZA | 42.40% | 52.22% | — | 0.277 |
| gliner-pii-edge | GLINER | 41.73% | 46.82% | — | 0.483 |
| ru-legal-ner | HF | 39.29% | 61.07% | — | 0.330 |
| ner-ru-gherman | HF | 36.48% | 60.13% | — | 0.478 |
| gliner-stream-pii | GLINER | 35.81% | 47.61% | — | 0.470 |
| gliner-pii-base | GLINER | 27.93% | 33.63% | — | 0.451 |
| fef2-secret-ru | HF | 11.71% | 15.60% | — | 0.274 |
| rules-ru | RULES | 11.12% | 11.55% | — | 0.322 |
| spacy-ru-lg | SPACY | 6.82% | 8.62% | — | 0.096 |
| natasha | NATASHA | 4.14% | 5.30% | — | 0.076 |
| spacy-alrosait | SPACY | 0.01% | 0.10% | — | 0.002 |