MULTI · PII
kiji-multi PII detection benchmark results
A frozen evaluation slice with 1,000 rows, 7,565 normalized gold annotations and source-level provenance.
1,000input rows
7,565gold annotations
447,991characters
58measured detectors
Dataset provenance
- Language group
- MULTI
- Task
- PII
- License
- apache-2.0
- Data categories
- ACCOUNT, ADDRESS, CONTACT, DATE, ID, NET, ORG, PERSON, SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| pplx+cpu-int8 | PPLX | 98.80% | 99.00% | — | 0.754 |
| nym-base | HF | 95.92% | 97.51% | — | 0.904 |
| nym-small | ONNX | 95.64% | 97.30% | — | 0.907 |
| pplx | PPLX | 95.62% | 95.80% | — | 0.865 |
| gliner2-fastino | GLINER2 | 94.71% | 95.78% | — | 0.871 |
| nym-base+cpu-int8 | HF | 94.38% | 96.59% | — | 0.903 |
| gliner2-hivetrace-omni | GLINER2 | 92.97% | 94.18% | — | 0.870 |
| bardsai-eu | ONNX | 90.96% | 95.14% | — | 0.842 |
| gliner-nvidia | GLINER | 90.64% | 93.23% | — | 0.856 |
| openmed-multilingual | HF | 79.50% | 83.30% | — | 0.825 |
| gliner2-large | GLINER2 | 78.84% | 80.24% | — | 0.803 |
| gliner-stream-pii | GLINER | 78.69% | 83.46% | — | 0.809 |
| openai-base-onnx | ONNX | 76.75% | 78.02% | — | 0.790 |
| openai-base | OPF | 76.66% | 77.83% | — | 0.789 |
| opf-kz-ru | OPF | 76.07% | 77.55% | — | 0.797 |
| apararti | OPF | 75.97% | 77.75% | — | 0.808 |
| mmbert32k | HF | 75.61% | 87.10% | — | 0.804 |
| nuner-zero | GLINER | 74.04% | 75.45% | — | 0.764 |
| gravitee-small | HF | 72.12% | 73.96% | — | 0.685 |
| kalyan-ettin | HF | 71.75% | 79.96% | — | 0.752 |
| gravitee-small+cpu-int8 | HF | 71.09% | 73.30% | — | 0.680 |
| gliner-multi-v21 | GLINER | 70.69% | 72.37% | — | 0.695 |
| gliner25-fastino | GLINER2 | 69.31% | 70.28% | — | 0.755 |
| openmed-nemotron | HF | 69.00% | 75.03% | — | 0.680 |
| gliner2-hivetrace-uni | GLINER2 | 67.85% | 68.86% | — | 0.757 |
| gliner-urchade | GLINER | 67.36% | 68.27% | — | 0.776 |
| opf-ru | OPF | 66.72% | 74.63% | — | 0.758 |
| traciora | OPF | 64.85% | 68.66% | — | 0.730 |
| pii-shield-onnx | ONNX | 62.49% | 68.16% | — | 0.381 |
| opf-ru-v2 | OPF | 60.22% | 64.07% | — | 0.733 |
| davlan-xlmr | HF | 57.62% | 58.94% | — | 0.625 |
| davlan-mbert | HF | 55.72% | 57.05% | — | 0.616 |
| davlan-mbert+cpu-int8 | HF | 55.31% | 56.44% | — | 0.611 |
| ner-ru-gherman | HF | 55.27% | 60.81% | — | 0.576 |
| ner-ru-gherman-onnx | ONNX | 54.95% | 60.49% | — | 0.572 |
| ru-pii-ner | RUPII | 53.88% | 54.78% | — | 0.662 |
| gliner-pii-edge | GLINER | 53.60% | 54.71% | — | 0.666 |
| gliner2-vladlinv | GLINER2 | 52.62% | 52.99% | — | 0.654 |
| ner-ru-gherman+cpu-int8 | HF | 52.07% | 56.38% | — | 0.538 |
| davlan-xlmr+cpu-int8 | HF | 51.45% | 54.21% | — | 0.592 |
| mmbert32k+cpu-int8 | HF | 50.89% | 67.36% | — | 0.653 |
| ner-ru-yqelz | HF | 49.76% | 52.20% | — | 0.563 |
| gliner-pii-base | GLINER | 49.19% | 50.01% | — | 0.664 |
| stanza-ru | STANZA | 44.52% | 46.33% | — | 0.302 |
| ru-legal-ner+cpu-int8 | HF | 43.37% | 53.99% | — | 0.253 |
| ru-legal-ner | HF | 43.32% | 53.97% | — | 0.253 |
| kalyan-ettin+cpu-int8 | HF | 41.57% | 48.39% | — | 0.528 |
| spacy-ru-lg | SPACY | 18.89% | 19.02% | — | 0.193 |
| gliner-pii-edge+cpu-int8 | GLINER | 17.48% | 18.77% | — | 0.294 |
| natasha | NATASHA | 9.58% | 9.62% | — | 0.135 |
| rules-ru | RULES | 6.65% | 6.75% | — | 0.269 |
| fef2-secret-ru | HF | 4.68% | 5.68% | — | 0.136 |
| fef2-secret-ru+cpu-int8 | HF | 3.58% | 4.72% | — | 0.103 |
| spacy-alrosait | SPACY | 0.11% | 0.11% | — | 0.001 |
| gliner-pii-base+cpu-int8 | GLINER | 0.04% | 0.04% | — | 0.001 |
| gliner-multi-v21+cpu-int8 | GLINER | 0.00% | 0.00% | — | 0.000 |
| gliner-nvidia+cpu-int8 | GLINER | 0.00% | 0.00% | — | 0.000 |
| gliner-urchade+cpu-int8 | GLINER | 0.00% | 0.00% | — | 0.000 |