EN · PII
privy PII detection benchmark results
A frozen evaluation slice with 1,500 rows, 1,899 normalized gold annotations and source-level provenance.
1,500input rows
1,899gold annotations
467,860characters
40measured detectors
Dataset provenance
- Language group
- EN
- Task
- PII
- License
- mit
- Data categories
- ADDRESS, CONTACT, ID, NET, ORG, PERSON, SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by fully hidden annotations. Training-source overlaps are excluded.
| Detector | Family | Fully hidden ↑ | Detected / any overlap ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| gravitee-small | HF | 99.21% | 99.58% | 1.91% | 0.878 |
| pplx | PPLX | 91.84% | 94.94% | 14.04% | 0.419 |
| nym-base | HF | 83.36% | 96.37% | 11.50% | 0.470 |
| nym-small | ONNX | 79.09% | 93.26% | 11.13% | 0.460 |
| gliner-nvidia | GLINER | 78.78% | 82.62% | 7.34% | 0.545 |
| gliner2-fastino | GLINER2 | 77.46% | 79.94% | 4.19% | 0.630 |
| nuner-zero | GLINER | 76.67% | 79.99% | 2.17% | 0.786 |
| gliner2-hivetrace-omni | GLINER2 | 73.99% | 75.57% | 5.72% | 0.548 |
| apararti | OPF | 69.25% | 75.57% | 10.42% | 0.419 |
| opf-kz-ru | OPF | 68.51% | 75.09% | 10.21% | 0.417 |
| gliner-stream-pii | GLINER | 66.14% | 68.04% | 0.93% | 0.746 |
| gliner2-large | GLINER2 | 64.56% | 68.62% | 4.31% | 0.562 |
| pii-shield-onnx | ONNX | 63.93% | 72.14% | 56.48% | 0.108 |
| openai-base | OPF | 63.82% | 68.83% | 8.68% | 0.418 |
| gliner25-fastino | GLINER2 | 61.19% | 63.03% | 2.77% | 0.623 |
| mmbert32k | HF | 60.40% | 77.62% | 6.08% | 0.558 |
| opf-ru | OPF | 59.87% | 74.72% | 11.62% | 0.362 |
| gliner-urchade | GLINER | 59.29% | 60.45% | 5.23% | 0.486 |
| gliner-pii-base | GLINER | 57.56% | 58.29% | 3.08% | 0.552 |
| gliner2-hivetrace-uni | GLINER2 | 56.82% | 58.82% | 6.31% | 0.408 |
| kalyan-ettin | HF | 55.45% | 74.67% | 4.79% | 0.547 |
| opf-ru-v2 | OPF | 51.13% | 60.45% | 8.16% | 0.385 |
| traciora | OPF | 49.82% | 59.87% | 8.04% | 0.378 |
| ru-pii-ner | RUPII | 49.29% | 54.29% | 3.28% | 0.474 |
| gliner-pii-edge | GLINER | 48.55% | 52.19% | 8.92% | 0.351 |
| openmed-nemotron | HF | 42.07% | 57.40% | 6.17% | 0.450 |
| gliner-multi-v21 | GLINER | 39.23% | 43.55% | 5.57% | 0.345 |
| ru-legal-ner | HF | 35.18% | 51.50% | 11.64% | 0.301 |
| bardsai-eu | ONNX | 35.02% | 48.87% | 6.10% | 0.419 |
| gliner2-vladlinv | GLINER2 | 33.33% | 33.81% | 1.63% | 0.349 |
| davlan-mbert | HF | 30.28% | 39.60% | 0.11% | 0.509 |
| davlan-xlmr | HF | 26.49% | 35.60% | 0.01% | 0.496 |
| ner-ru-yqelz | HF | 23.49% | 34.39% | 1.94% | 0.440 |
| ner-ru-gherman | HF | 20.59% | 34.60% | 0.10% | 0.386 |
| rules-ru | RULES | 15.22% | 15.38% | 0.06% | 0.384 |
| stanza-ru | STANZA | 11.11% | 20.06% | 8.32% | 0.173 |
| natasha | NATASHA | 8.85% | 9.37% | 3.93% | 0.090 |
| fef2-secret-ru | HF | 6.42% | 7.53% | 0.75% | 0.155 |
| spacy-ru-lg | SPACY | 3.21% | 4.27% | 3.38% | 0.052 |
| spacy-alrosait | SPACY | 0.00% | 0.00% | 0.00% | 0.000 |