STANZA detector
stanza-ru PII detection benchmark
Measured on 41 of 41 retained datasets under the same frozen scoring and masking protocol.
43.41%fully hidden annotations
51.15%detected / overlapped
16.11%extra masking
1,198CPU characters / second
Strengths
Highest complete-masking rates by normalized data type.
People's names84.78%55,730 / 65,736
Organizations68.35%13,174 / 19,274
Addresses & locations56.21%22,262 / 39,602
Watch list
Lowest observed complete-masking rates; inspect before deployment.
Dates & times0.98%54 / 5,502
Network identifiers1.63%417 / 25,631
Logins & usernames4.02%457 / 11,382
Language coverage
The overall leader can differ from the best choice for one language slice.
| Language | Datasets | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| English | 17 | 51.77% | 58.01% | 18.35% |
| Russian | 19 | 41.90% | 50.10% | 10.69% |
| Multilingual | 5 | 41.37% | 48.99% | 20.42% |
Configuration, source and version
- Family
- STANZA
- Revision
- dcbf023decb5a319fff84780779052ed974b9013
- Eligible datasets
- 41 / 41
- Character F1
- 0.326
- Untouched diagnostic
- 48.85% · 111,122 / 227,466
- Benchmark version
- 1.0.3 · 2026-09-09
- CPU seconds / 10k chars
- 8.347
- GPU seconds / 10k chars
- Not measured
Results by dataset
Complete masking is primary; detection shows whether any character overlapped.
| Dataset | Gold annotations | Fully hidden ↑ | Detected ↑ | Extra masking ↓ |
|---|---|---|---|---|
| rubai-ru | 3,458 | 5.44% | 62.17% | — |
| multiconer-ru | 1,208 | 7.12% | 10.02% | 2.89% |
| secrets-rules | 746 | 9.92% | 19.97% | 44.34% |
| privy | 1,899 | 11.11% | 20.06% | 8.32% |
| synth-ru-tickets | 32,099 | 11.73% | 16.47% | — |
| creddata | 776 | 12.37% | 20.36% | 15.22% |
| synth-wiki-tables | 13,094 | 13.69% | 15.95% | — |
| leaky-repo | 95 | 14.74% | 25.26% | 11.30% |
| leak-museum | 101 | 14.85% | 28.71% | 20.40% |
| nym-multi | 8,169 | 16.55% | 21.13% | 11.24% |
| synth-env-configs | 392 | 18.62% | 48.72% | 22.58% |
| secrets-issues | 288 | 19.79% | 37.85% | 17.39% |
| synth-secrets-en | 581 | 20.65% | 45.09% | 15.70% |
| corrupt-secrets-issues | 288 | 20.83% | 47.22% | 20.76% |
| corrupt-hivetrace | 1,667 | 21.30% | 32.75% | 13.55% |
| alexen2 | 1,261 | 21.33% | 51.15% | 14.94% |
| synth-jira-comments | 2,705 | 22.85% | 26.36% | — |
| hivetrace | 1,667 | 22.98% | 31.73% | 1.84% |
| synth-secrets-ru | 581 | 28.40% | 59.04% | 14.83% |
| arthur-passwords | 280 | 31.07% | 31.07% | 26.94% |
| nym-ru | 9,497 | 33.41% | 45.61% | — |
| ameau01 | 2,242 | 37.73% | 51.43% | 16.65% |
| russian-pii-66k | 4,805 | 39.13% | 49.57% | — |
| corrupt-redmadrobot | 5,531 | 40.14% | 48.92% | 14.14% |
| redact-multi | 22,864 | 42.40% | 52.22% | — |
| redmadrobot | 5,516 | 42.98% | 49.46% | 9.48% |
| nym-en | 3,584 | 43.78% | 51.40% | 9.41% |
| kiji-multi | 7,565 | 44.52% | 46.33% | — |
| tonicai | 2,417 | 44.60% | 45.01% | 3.36% |
| nemotron-pii | 9,391 | 47.07% | 52.15% | 42.47% |
| dialogpii-multi | 10,345 | 49.77% | 54.96% | — |
| jayguard | 1,195 | 51.55% | 73.97% | 8.92% |
| redact-ru | 9,230 | 52.13% | 63.29% | 10.37% |
| alrosait | 1,862 | 54.83% | 95.81% | 6.24% |
| gretel-multi | 4,484 | 56.67% | 74.00% | 20.56% |
| scanpatch | 8,708 | 57.82% | 65.87% | 3.64% |
| kiji-en | 7,623 | 70.38% | 72.64% | — |
| tab-echr | 3,830 | 77.26% | 83.99% | — |
| dialogpii-en | 3,087 | 78.81% | 82.99% | 40.89% |
| nerel | 24,369 | 85.35% | 89.57% | — |
| factrueval | 7,966 | 95.56% | 96.72% | — |