RUPII detector
ru-pii-ner PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
53.11%fully hidden annotations
57.30%detected / overlapped
2.73%extra masking
7,053CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Dates & times84.22%4,634 / 5,502
Documents & identifiers81.82%12,301 / 15,034
Phone numbers & email80.97%17,797 / 21,981
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Organizations2.95%568 / 19,274
Other sensitive attributes8.73%280 / 3,207
Network identifiers9.12%2,337 / 25,631
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 53.30% | 56.47% | 2.43% |
| Russian | 19 | 53.34% | 58.32% | 3.50% |
| Multilingual | 5 | 52.39% | 55.30% | 0.66% |
Configuration, source and version
- Family
- RUPII
- Revision
- 3b509ebbcec894e522adef32560644017832deea
- Eligible datasets
- 41 / 41
- Character F1
- 0.599
- Untouched diagnostic
- 42.70% · 97,121 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 1.418
- GPU seconds / 10k chars
- Not measured
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| leaky-repo | 95 | 5.26% | 11.58% | 0.09% |
| secrets-issues | 288 | 7.29% | 12.15% | 1.26% |
| rubai-ru | 3,458 | 8.01% | 94.88% | — |
| secrets-rules | 746 | 9.92% | 14.34% | 11.96% |
| synth-env-configs | 392 | 10.20% | 12.24% | 4.32% |
| synth-secrets-en | 581 | 10.67% | 14.63% | 4.62% |
| leak-museum | 101 | 11.88% | 16.83% | 0.76% |
| synth-secrets-ru | 581 | 12.91% | 17.90% | 5.05% |
| tab-echr | 3,830 | 15.74% | 30.34% | — |
| corrupt-secrets-issues | 288 | 19.44% | 26.39% | 1.74% |
| multiconer-ru | 1,208 | 25.50% | 34.69% | 3.66% |
| alexen2 | 1,261 | 26.33% | 100.00% | 6.89% |
| factrueval | 7,966 | 29.79% | 29.84% | — |
| nerel | 24,369 | 32.84% | 33.49% | — |
| creddata | 776 | 34.54% | 38.92% | 2.17% |
| dialogpii-multi | 10,345 | 35.68% | 39.55% | — |
| synth-ru-tickets | 32,099 | 39.14% | 41.29% | — |
| synth-jira-comments | 2,705 | 43.81% | 44.10% | — |
| ameau01 | 2,242 | 44.47% | 44.74% | 1.09% |
| tonicai | 2,417 | 48.90% | 51.26% | 0.88% |
| privy | 1,899 | 49.29% | 54.29% | 3.28% |
| dialogpii-en | 3,087 | 53.74% | 57.95% | 8.08% |
| kiji-multi | 7,565 | 53.88% | 54.78% | — |
| redact-multi | 22,864 | 55.93% | 59.06% | — |
| gretel-multi | 4,484 | 55.93% | 60.53% | 0.61% |
| arthur-passwords | 280 | 56.79% | 56.79% | 10.85% |
| nym-multi | 8,169 | 60.29% | 62.32% | 3.56% |
| nemotron-pii | 9,391 | 65.20% | 65.99% | 1.03% |
| scanpatch | 8,708 | 65.94% | 67.88% | 1.33% |
| synth-wiki-tables | 13,094 | 66.67% | 71.25% | — |
| kiji-en | 7,623 | 66.84% | 67.60% | — |
| jayguard | 1,195 | 67.78% | 79.08% | 1.45% |
| corrupt-redmadrobot | 5,531 | 72.93% | 78.50% | 5.89% |
| redact-ru | 9,230 | 75.20% | 77.70% | 8.39% |
| redmadrobot | 5,516 | 76.63% | 79.88% | 2.63% |
| nym-en | 3,584 | 77.04% | 78.93% | 1.84% |
| corrupt-hivetrace | 1,667 | 83.32% | 88.84% | 0.71% |
| nym-ru | 9,497 | 83.39% | 84.29% | — |
| hivetrace | 1,667 | 88.42% | 90.04% | 0.41% |
| alrosait | 1,862 | 94.20% | 95.97% | 2.23% |
| russian-pii-66k | 4,805 | 96.23% | 96.36% | — |