EN · PII
privy PII detection benchmark results
A frozen evaluation slice with 1,500 rows, 1,899 normalized gold annotations and source-level provenance.
1,500input rows
1,899gold annotations
467,860characters
40measured detectors
Dataset provenance
- Language group
- EN
- Task
- PII
- License
- mit
- Data categories
- ADDRESS, CONTACT, ID, NET, ORG, PERSON, SECRET
- Project generated
- No
- Corrupted copy
- No
Rows without annotations are not verified clean. Reported extra masking is not a human-confirmed false-positive rate.
Detector outcomes
Sorted by untouched annotations. Training-source overlaps are excluded.
| Detector | Family | Untouched ↓ | Fully hidden ↑ | Extra masking ↓ | Char F1 ↑ |
|---|---|---|---|---|---|
| gravitee-small | HF | 0.42% | 99.21% | 1.91% | 0.878 |
| nym-base | HF | 3.63% | 83.36% | 11.50% | 0.470 |
| pplx | PPLX | 5.06% | 91.84% | 14.04% | 0.419 |
| nym-small | ONNX | 6.74% | 79.09% | 11.13% | 0.460 |
| gliner-nvidia | GLINER | 17.38% | 78.78% | 7.34% | 0.545 |
| nuner-zero | GLINER | 20.01% | 76.67% | 2.17% | 0.786 |
| gliner2-fastino | GLINER2 | 20.06% | 77.46% | 4.19% | 0.630 |
| mmbert32k | HF | 22.38% | 60.40% | 6.08% | 0.558 |
| gliner2-hivetrace-omni | GLINER2 | 24.43% | 73.99% | 5.72% | 0.548 |
| apararti | OPF | 24.43% | 69.25% | 10.42% | 0.419 |
| opf-kz-ru | OPF | 24.91% | 68.51% | 10.21% | 0.417 |
| opf-ru | OPF | 25.28% | 59.87% | 11.62% | 0.362 |
| kalyan-ettin | HF | 25.33% | 55.45% | 4.79% | 0.547 |
| pii-shield-onnx | ONNX | 27.86% | 63.93% | 56.48% | 0.108 |
| openai-base | OPF | 31.17% | 63.82% | 8.68% | 0.418 |
| gliner2-large | GLINER2 | 31.38% | 64.56% | 4.31% | 0.562 |
| gliner-stream-pii | GLINER | 31.96% | 66.14% | 0.93% | 0.746 |
| gliner25-fastino | GLINER2 | 36.97% | 61.19% | 2.77% | 0.623 |
| gliner-urchade | GLINER | 39.55% | 59.29% | 5.23% | 0.486 |
| opf-ru-v2 | OPF | 39.55% | 51.13% | 8.16% | 0.385 |
| traciora | OPF | 40.13% | 49.82% | 8.04% | 0.378 |
| gliner2-hivetrace-uni | GLINER2 | 41.18% | 56.82% | 6.31% | 0.408 |
| gliner-pii-base | GLINER | 41.71% | 57.56% | 3.08% | 0.552 |
| openmed-nemotron | HF | 42.60% | 42.07% | 6.17% | 0.450 |
| ru-pii-ner | RUPII | 45.71% | 49.29% | 3.28% | 0.474 |
| gliner-pii-edge | GLINER | 47.81% | 48.55% | 8.92% | 0.351 |
| ru-legal-ner | HF | 48.50% | 35.18% | 11.64% | 0.301 |
| bardsai-eu | ONNX | 51.13% | 35.02% | 6.10% | 0.419 |
| gliner-multi-v21 | GLINER | 56.45% | 39.23% | 5.57% | 0.345 |
| davlan-mbert | HF | 60.40% | 30.28% | 0.11% | 0.509 |
| davlan-xlmr | HF | 64.40% | 26.49% | 0.01% | 0.496 |
| ner-ru-gherman | HF | 65.40% | 20.59% | 0.10% | 0.386 |
| ner-ru-yqelz | HF | 65.61% | 23.49% | 1.94% | 0.440 |
| gliner2-vladlinv | GLINER2 | 66.19% | 33.33% | 1.63% | 0.349 |
| stanza-ru | STANZA | 79.94% | 11.11% | 8.32% | 0.173 |
| rules-ru | RULES | 84.62% | 15.22% | 0.06% | 0.384 |
| natasha | NATASHA | 90.63% | 8.85% | 3.93% | 0.090 |
| fef2-secret-ru | HF | 92.47% | 6.42% | 0.75% | 0.155 |
| spacy-ru-lg | SPACY | 95.73% | 3.21% | 3.38% | 0.052 |
| spacy-alrosait | SPACY | 100.00% | 0.00% | 0.00% | 0.000 |