RULES detector
rules-ru PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
24.48%fully hidden annotations
25.37%detected / overlapped
6.96%extra masking
—CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
Network identifiers93.39%23,938 / 25,631
Phone numbers & email81.10%17,826 / 21,981
Passwords, keys & tokens47.34%3,252 / 6,870
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Dates & times0.05%3 / 5,502
Addresses & locations0.09%36 / 39,602
People's names0.13%88 / 65,736
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 15.39% | 16.51% | 8.90% |
| Russian | 19 | 33.12% | 34.12% | 2.74% |
| Multilingual | 5 | 8.85% | 9.27% | 0.04% |
Configuration, source and version
- Family
- RULES
- Revision
- 1.0
- Eligible datasets
- 41 / 41
- Character F1
- 0.506
- Untouched diagnostic
- 74.63% · 169,760 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- Not measured
- GPU seconds / 10k chars
- Not measured
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| alrosait | 1,862 | 0.00% | 0.00% | 2.46% |
| tab-echr | 3,830 | 0.00% | 0.00% | — |
| jayguard | 1,195 | 0.08% | 0.08% | 0.84% |
| nerel | 24,369 | 0.12% | 0.15% | — |
| rubai-ru | 3,458 | 0.17% | 3.44% | — |
| multiconer-ru | 1,208 | 0.25% | 0.50% | 0.00% |
| dialogpii-en | 3,087 | 0.32% | 3.56% | 0.00% |
| factrueval | 7,966 | 0.55% | 0.55% | — |
| dialogpii-multi | 10,345 | 2.14% | 2.81% | — |
| alexen2 | 1,261 | 4.68% | 54.72% | 0.00% |
| kiji-multi | 7,565 | 6.65% | 6.75% | — |
| kiji-en | 7,623 | 7.84% | 8.13% | — |
| arthur-passwords | 280 | 8.93% | 8.93% | 5.10% |
| corrupt-redmadrobot | 5,531 | 9.65% | 11.95% | 0.18% |
| nym-multi | 8,169 | 9.95% | 10.42% | 0.00% |
| nym-en | 3,584 | 10.44% | 10.69% | 0.00% |
| redact-multi | 22,864 | 11.12% | 11.55% | — |
| scanpatch | 8,708 | 11.66% | 14.54% | 0.18% |
| gretel-multi | 4,484 | 14.41% | 14.72% | 0.04% |
| redmadrobot | 5,516 | 14.58% | 15.57% | 0.18% |
| privy | 1,899 | 15.22% | 15.38% | 0.06% |
| tonicai | 2,417 | 15.56% | 19.16% | 0.00% |
| ameau01 | 2,242 | 16.32% | 16.32% | 3.68% |
| redact-ru | 9,230 | 18.02% | 18.73% | 2.44% |
| nemotron-pii | 9,391 | 18.06% | 18.75% | 0.00% |
| nym-ru | 9,497 | 20.83% | 21.10% | — |
| russian-pii-66k | 4,805 | 33.63% | 33.86% | — |
| leaky-repo | 95 | 34.74% | 36.84% | 4.49% |
| creddata | 776 | 38.02% | 39.69% | 12.01% |
| corrupt-secrets-issues | 288 | 40.97% | 59.72% | 7.35% |
| corrupt-hivetrace | 1,667 | 41.57% | 45.77% | 0.00% |
| secrets-issues | 288 | 48.96% | 67.71% | 10.19% |
| leak-museum | 101 | 61.39% | 63.37% | 1.34% |
| hivetrace | 1,667 | 64.31% | 64.31% | 0.00% |
| synth-wiki-tables | 13,094 | 69.70% | 69.70% | — |
| synth-jira-comments | 2,705 | 70.17% | 70.17% | — |
| secrets-rules | 746 | 74.53% | 75.60% | 32.13% |
| synth-ru-tickets | 32,099 | 75.27% | 75.27% | — |
| synth-secrets-ru | 581 | 81.41% | 81.93% | 16.74% |
| synth-secrets-en | 581 | 81.76% | 82.27% | 16.69% |
| synth-env-configs | 392 | 95.66% | 95.92% | 11.63% |