MULTI · PII
kiji-multi PII detection benchmark results
A frozen evaluation slice with 1,000 rows, 7,565 normalized gold annotations and source-level provenance.
1,000input rows
7,565gold annotations
447,991characters
58measured detectors
Dataset provenance
- Language group
- MULTI
- Task
- PII
- License
- apache-2.0
- Data categories
- ACCOUNT, ADDRESS, CONTACT, DATE, ID, NET, ORG, PERSON, SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by untouched annotations. Training-source overlaps are excluded.
| Detector | Family | Untouched ↓ | Fully hidden ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pplx+cpu-int8 | PPLX | 1.00% | 98.80% | — | 0.754 |
| nym-base | HF | 2.49% | 95.92% | — | 0.904 |
| nym-small | ONNX | 2.70% | 95.64% | — | 0.907 |
| nym-base+cpu-int8 | HF | 3.41% | 94.38% | — | 0.903 |
| pplx | PPLX | 4.20% | 95.62% | — | 0.865 |
| gliner2-fastino | GLINER2 | 4.22% | 94.71% | — | 0.871 |
| bardsai-eu | ONNX | 4.86% | 90.96% | — | 0.842 |
| gliner2-hivetrace-omni | GLINER2 | 5.82% | 92.97% | — | 0.870 |
| gliner-nvidia | GLINER | 6.77% | 90.64% | — | 0.856 |
| mmbert32k | HF | 12.90% | 75.61% | — | 0.804 |
| gliner-stream-pii | GLINER | 16.54% | 78.69% | — | 0.809 |
| openmed-multilingual | HF | 16.70% | 79.50% | — | 0.825 |
| gliner2-large | GLINER2 | 19.76% | 78.84% | — | 0.803 |
| kalyan-ettin | HF | 20.04% | 71.75% | — | 0.752 |
| openai-base-onnx | ONNX | 21.98% | 76.75% | — | 0.790 |
| openai-base | OPF | 22.17% | 76.66% | — | 0.789 |
| apararti | OPF | 22.25% | 75.97% | — | 0.808 |
| opf-kz-ru | OPF | 22.45% | 76.07% | — | 0.797 |
| nuner-zero | GLINER | 24.55% | 74.04% | — | 0.764 |
| openmed-nemotron | HF | 24.97% | 69.00% | — | 0.680 |
| opf-ru | OPF | 25.37% | 66.72% | — | 0.758 |
| gravitee-small | HF | 26.04% | 72.12% | — | 0.685 |
| gravitee-small+cpu-int8 | HF | 26.70% | 71.09% | — | 0.680 |
| gliner-multi-v21 | GLINER | 27.63% | 70.69% | — | 0.695 |
| gliner25-fastino | GLINER2 | 29.72% | 69.31% | — | 0.755 |
| gliner2-hivetrace-uni | GLINER2 | 31.14% | 67.85% | — | 0.757 |
| traciora | OPF | 31.34% | 64.85% | — | 0.730 |
| gliner-urchade | GLINER | 31.73% | 67.36% | — | 0.776 |
| pii-shield-onnx | ONNX | 31.84% | 62.49% | — | 0.381 |
| mmbert32k+cpu-int8 | HF | 32.64% | 50.89% | — | 0.653 |
| opf-ru-v2 | OPF | 35.93% | 60.22% | — | 0.733 |
| ner-ru-gherman | HF | 39.19% | 55.27% | — | 0.576 |
| ner-ru-gherman-onnx | ONNX | 39.51% | 54.95% | — | 0.572 |
| davlan-xlmr | HF | 41.06% | 57.62% | — | 0.625 |
| davlan-mbert | HF | 42.95% | 55.72% | — | 0.616 |
| davlan-mbert+cpu-int8 | HF | 43.56% | 55.31% | — | 0.611 |
| ner-ru-gherman+cpu-int8 | HF | 43.62% | 52.07% | — | 0.538 |
| ru-pii-ner | RUPII | 45.22% | 53.88% | — | 0.662 |
| gliner-pii-edge | GLINER | 45.29% | 53.60% | — | 0.666 |
| davlan-xlmr+cpu-int8 | HF | 45.79% | 51.45% | — | 0.592 |
| ru-legal-ner+cpu-int8 | HF | 46.01% | 43.37% | — | 0.253 |
| ru-legal-ner | HF | 46.03% | 43.32% | — | 0.253 |
| gliner2-vladlinv | GLINER2 | 47.01% | 52.62% | — | 0.654 |
| ner-ru-yqelz | HF | 47.80% | 49.76% | — | 0.563 |
| gliner-pii-base | GLINER | 49.99% | 49.19% | — | 0.664 |
| kalyan-ettin+cpu-int8 | HF | 51.61% | 41.57% | — | 0.528 |
| stanza-ru | STANZA | 53.67% | 44.52% | — | 0.302 |
| spacy-ru-lg | SPACY | 80.98% | 18.89% | — | 0.193 |
| gliner-pii-edge+cpu-int8 | GLINER | 81.23% | 17.48% | — | 0.294 |
| natasha | NATASHA | 90.38% | 9.58% | — | 0.135 |
| rules-ru | RULES | 93.25% | 6.65% | — | 0.269 |
| fef2-secret-ru | HF | 94.32% | 4.68% | — | 0.136 |
| fef2-secret-ru+cpu-int8 | HF | 95.28% | 3.58% | — | 0.103 |
| spacy-alrosait | SPACY | 99.89% | 0.11% | — | 0.001 |
| gliner-pii-base+cpu-int8 | GLINER | 99.96% | 0.04% | — | 0.001 |
| gliner-multi-v21+cpu-int8 | GLINER | 100.00% | 0.00% | — | 0.000 |
| gliner-nvidia+cpu-int8 | GLINER | 100.00% | 0.00% | — | 0.000 |
| gliner-urchade+cpu-int8 | GLINER | 100.00% | 0.00% | — | 0.000 |