Open source secret detection benchmark
Compare secret scanners and detection models on retained secrets datasets using the same masking and scoring protocol.
ScopeAll languagesSecrets datasets10 datasets4,128 normalized annotationsPooled annotations2026-09-19
Start with what you need to protect
Compare personal data, secrets, and the datasets where detectors struggle.
Performance from easy to extreme
Dataset average · see how much sensitive data each detector hides completely at every difficulty level. Higher is better.
| Detector | Easy | Medium | Hard | Extreme |
|---|---|---|---|---|
| 96.2% | 90.7% | 87.7% | — | |
| 93.3% | 93.4% | 82.8% | — | |
| 97.1% | 87.1% | 75.8% | — | |
| 97.2% | 83.4% | 75.0% | — | |
| 93.1% | 78.7% | 68.2% | — | |
| 96.7% | 79.0% | 72.1% | — | |
| 94.1% | 72.1% | 67.3% | — | |
| 94.1% | 69.6% | 67.9% | — | |
| 89.4% | 64.1% | 60.0% | — | |
| 89.5% | 64.6% | 55.9% | — | |
| 83.3% | 71.2% | 59.9% | — | |
| 80.6% | 65.8% | 56.8% | — | |
| 87.4% | 59.4% | 53.3% | — | |
| 82.6% | 60.2% | 49.9% | — | |
| 90.3% | 54.8% | 37.4% | — | |
| 80.6% | 57.7% | 42.8% | — | |
| 72.8% | 55.1% | 53.8% | — | |
| 40.5% | 66.0% | 61.7% | — | |
| 74.5% | 47.2% | 49.8% | — | |
| 76.0% | 52.0% | 48.0% | — | |
| 75.6% | 41.3% | 41.4% | — | |
| 55.5% | 48.0% | 43.8% | — | |
| 71.1% | 33.9% | 37.6% | — | |
| 55.4% | 52.9% | 33.7% | — | |
| 65.0% | 34.0% | 37.9% | — | |
| 66.4% | 25.9% | 31.9% | — | |
| 55.2% | 31.7% | 22.9% | — | |
| 64.5% | 23.7% | 12.5% | — | |
| 44.8% | 24.7% | 18.2% | — | |
| 46.1% | 21.4% | 13.0% | — | |
| 38.8% | 18.3% | 15.9% | — | |
| 33.7% | 9.6% | 12.2% | — | |
| 17.8% | 21.3% | 18.5% | — | |
| 21.9% | 8.2% | 13.2% | — | |
| 17.2% | 6.2% | 8.1% | — | |
| 6.0% | 1.6% | 3.1% | — | |
| 0.0% | 0.1% | 1.2% | — | |
| 0.4% | 0.0% | 0.1% | — | |
| 0.3% | 0.0% | 0.2% | — | |
| 0.0% | 0.0% | 0.0% | — | |
| 0.0% | 0.0% | 0.0% | — | |
pplx+ov100Not all datasets | 86.6% | — | 80.9% | — |
opf-ru-v2+ov100Not all datasets | 91.0% | — | 66.8% | — |
opf-ru-v2+sent300Not all datasets | 90.6% | — | 67.6% | — |
pplx+sent300Not all datasets | 85.8% | — | 77.9% | — |
pplx+cpu-int8Not all datasets | — | — | 81.3% | — |
nym-base+sent300Not all datasets | 87.2% | — | 53.6% | — |
nym-base+ov100Not all datasets | 87.4% | — | 52.6% | — |
pplx+homoglyphNot all datasets | — | — | 75.7% | — |
credsweeper-nomlNot all datasets | 82.2% | 54.7% | 60.1% | — |
openai-base-onnxNot all datasets | — | — | 70.8% | — |
credsweeperNot all datasets | 76.4% | — | 55.9% | — |
opf-ru-v2+homoglyphNot all datasets | — | — | 70.1% | — |
nym-base+cpu-int8Not all datasets | — | — | 69.4% | — |
gliner-nvidia+ov100Not all datasets | 78.6% | — | 50.6% | — |
gliner25-fastino+ov100Not all datasets | 82.1% | — | 45.0% | — |
gliner-nvidia+sent300Not all datasets | 78.6% | — | 49.4% | — |
gliner25-fastino+sent300Not all datasets | 82.3% | — | 40.9% | — |
gliner25-fastino+nochunkNot all datasets | 82.1% | — | 35.3% | — |
nym-base+homoglyphNot all datasets | — | — | 63.2% | — |
ru-legal-ner+ov100Not all datasets | 68.2% | — | 41.2% | — |
ru-legal-ner+sent300Not all datasets | 67.7% | — | 39.5% | — |
mmbert32k+cpu-int8Not all datasets | — | — | 55.9% | — |
mmbert32k+nochunkNot all datasets | 60.4% | — | 34.2% | — |
ru-legal-ner+cpu-int8Not all datasets | — | — | 53.5% | — |
kalyan-ettin+cpu-int8Not all datasets | — | — | 53.1% | — |
gliner25-fastino-ruNot all datasets | — | 52.0% | — | — |
gliner2-fastino-ruNot all datasets | — | 46.5% | — | — |
betterleaksNot all datasets | — | — | 44.8% | — |
gliner-nvidia-ruNot all datasets | — | 42.3% | — | — |
gliner2-hivetrace-omni-ruNot all datasets | — | 41.7% | — | — |
ru-legal-ner+homoglyphNot all datasets | — | — | 41.3% | — |
gitleaksNot all datasets | — | 23.2% | 45.5% | — |
detect-secretsNot all datasets | 45.7% | 32.6% | 24.3% | — |
deepsecretsNot all datasets | 40.9% | 28.4% | 34.7% | — |
gliner2-vladlinv-ruNot all datasets | — | 33.2% | — | — |
fef2-secret-ru+cpu-int8Not all datasets | — | — | 30.9% | — |
gliner-nvidia+homoglyphNot all datasets | — | — | 29.9% | — |
titusNot all datasets | — | — | 24.3% | — |
trufflehogNot all datasets | 23.6% | — | 12.5% | — |
gliner-urchade-ruNot all datasets | — | 20.5% | — | — |
noseyparkerNot all datasets | 16.2% | — | 23.3% | — |
gliner-multi-v21-ruNot all datasets | — | 16.2% | — | — |
kingfisherNot all datasets | — | — | 13.9% | — |
gliner2-hivetrace-uni-ruNot all datasets | — | 11.7% | — | — |
gravitee-small+cpu-int8Not all datasets | — | — | 11.5% | — |
gliner-pii-edge+cpu-int8Not all datasets | — | — | 11.1% | — |
gliner2-vladlinv+homoglyphNot all datasets | — | — | 4.2% | — |
davlan-mbert+cpu-int8Not all datasets | — | — | 0.0% | — |
davlan-xlmr+cpu-int8Not all datasets | — | — | 0.0% | — |
gliner-multi-v21+cpu-int8Not all datasets | — | — | 0.0% | — |
gliner-nvidia+cpu-int8Not all datasets | — | — | 0.0% | — |
gliner-pii-base+cpu-int8Not all datasets | — | — | 0.0% | — |
gliner-urchade+cpu-int8Not all datasets | — | — | 0.0% | — |
ner-ru-gherman-onnxNot all datasets | — | — | 0.0% | — |
ner-ru-gherman+cpu-int8Not all datasets | — | — | 0.0% | — |
ner-ru-gherman+homoglyphNot all datasets | — | — | 0.0% | — |
PII datasets vs secrets datasets
Pooled annotations · compare how well each detector hides labeled items in datasets assigned to each task.
DetectorPII datasetsSecrets datasets
Use one detector or combine several?
See what combining detectors adds and how much extra text it masks.
One detector vs several combined
See whether combining detectors hides more sensitive data and how much extra text it masks.
| Detector setup | Completely hidden, % ↑ 0%50%100% | Extra text masked, % ↓ | Not fully hidden, items ↓ |
|---|---|---|---|
| Four detectorsFastino + PPLX + BardsAI EU + mmBERT | 195 | ||
| CPU + GPUFastino + PPLX + BardsAI EU | 203 | ||
| Three detectorsFastino + PPLX + mmBERT | 234 | ||
| Two detectorsFastino + PPLX | 250 | ||
| Single detectorPPLX | 363 | ||
| Single detectorGLiNER2 Fastino | 1,407 |
Single detectorRuns on CUDACPU + GPU
Find the best detector for each data type
Focus on the sensitive information your application needs to protect.
Hardest datasets
Higher scores mean the dataset was harder for the tested detectors.
Best detector by data type
Pooled annotations by annotation type · choose what sensitive data you need to hide.
Detector leaderboard
41 detector setups · 10 datasets · sorted by fully hidden descending
| Compare | Rank | Family | |||||||
|---|---|---|---|---|---|---|---|---|---|
| 01 | pii-shield-onnx | ONNX | 94.21% | 98.35% | 49.39% | 0.222 | — | — | |
| 02 | pplx | PPLX | 91.21% | 97.02% | 16.09% | 0.419 | 9.94 | 2.20 | |
| 03 | opf-kz-ru | OPF | 90.24% | 97.41% | 15.20% | 0.447 | 3.39 | 1.59 | |
| 04 | apararti | OPF | 88.71% | 97.48% | 15.32% | 0.450 | 4.09 | 1.69 | |
| 05 | openai-base | OPF | 85.25% | 93.12% | 12.38% | 0.484 | 4.21 | 1.80 | |
| 06 | opf-ru | OPF | 84.18% | 98.13% | 12.23% | 0.495 | 3.52 | 1.12 | |
| 07 | opf-ru-v2 | OPF | 82.49% | 95.40% | 9.51% | 0.528 | 3.47 | — | |
| 08 | traciora | OPF | 81.13% | 96.10% | 10.12% | 0.528 | 3.29 | 1.36 | |
| 09 | gliner-pii-edge | GLINER | 74.59% | 87.86% | 12.82% | 0.436 | 6.88 | 5.80 | |
| 10 | nym-base | HF | 73.40% | 95.71% | 12.57% | 0.456 | 3.21 | 0.14 | |
| 11 | openmed-multilingual | HF | 73.35% | 93.12% | 16.11% | 0.408 | 3.25 | 0.84 | |
| 12 | bardsai-eu | ONNX | 70.95% | 87.35% | 10.80% | 0.486 | — | — | |
| 13 | nym-small | ONNX | 69.21% | 95.16% | 13.62% | 0.443 | — | — | |
| 14 | gliner2-fastino | GLINER2 | 65.92% | 75.19% | 7.56% | 0.456 | 14.93 | 0.27 | |
| 15 | gliner25-fastino | GLINER2 | 64.53% | 69.82% | 10.06% | 0.309 | 12.47 | 0.15 | |
| 16 | gliner-nvidia | GLINER | 62.23% | 66.57% | 5.71% | 0.493 | 22.62 | 0.37 | |
| 17 | openmed-nemotron | HF | 62.09% | 86.07% | 16.55% | 0.399 | 3.04 | 0.87 | |
| 18 | rules-ru | RULES | 61.85% | 65.24% | 10.67% | 0.433 | — | — | |
| 19 | kalyan-ettin | HF | 58.45% | 86.63% | 10.63% | 0.457 | 1.14 | — | |
| 20 | nuner-zero | GLINER | 58.09% | 68.85% | 4.57% | 0.437 | 20.12 | 0.38 | |
| 21 | gliner-stream-pii | GLINER | 56.23% | 66.93% | 4.17% | 0.505 | 8.20 | 0.21 | |
| 22 | fef2-secret-ru | HF | 51.28% | 64.17% | 2.80% | 0.513 | 3.05 | 0.15 | |
| 23 | ru-legal-ner | HF | 50.48% | 87.52% | 14.32% | 0.351 | 0.20 | 0.13 | |
| 24 | gliner2-hivetrace-omni | GLINER2 | 44.33% | 47.04% | 3.05% | 0.337 | 14.79 | 0.29 | |
| 25 | gliner-urchade | GLINER | 44.06% | 49.42% | 4.16% | 0.317 | 9.68 | 0.20 | |
| 26 | mmbert32k | HF | 42.51% | 83.48% | 14.45% | 0.363 | 2.32 | 0.18 | |
| 27 | gravitee-small | HF | 39.03% | 55.94% | 5.95% | 0.282 | 0.66 | — | |
| 28 | gliner2-vladlinv | GLINER2 | 36.26% | 38.91% | 1.11% | 0.260 | 14.49 | 0.17 | |
| 29 | gliner2-large | GLINER2 | 24.93% | 29.36% | 3.48% | 0.139 | 24.69 | 0.61 | |
| 30 | gliner-pii-base | GLINER | 24.81% | 27.18% | 1.85% | 0.192 | 5.24 | 0.16 | |
| 31 | gliner-multi-v21 | GLINER | 19.55% | 25.46% | 2.65% | 0.102 | 9.89 | 0.19 | |
| 32 | ru-pii-ner | RUPII | 18.70% | 22.87% | 2.66% | 0.208 | 1.42 | — | |
| 33 | stanza-ru | STANZA | 18.44% | 36.05% | 19.31% | 0.202 | 8.35 | — | |
| 34 | ner-ru-yqelz | HF | 17.03% | 25.39% | 9.92% | 0.146 | 7.08 | 0.18 | |
| 35 | gliner2-hivetrace-uni | GLINER2 | 9.88% | 11.41% | 0.67% | 0.088 | 2.92 | — | |
| 36 | spacy-ru-lg | SPACY | 4.09% | 4.58% | 2.87% | 0.230 | 0.19 | — | |
| 37 | natasha | NATASHA | 0.17% | 0.51% | 0.76% | 0.065 | 0.08 | — | |
| 38 | ner-ru-gherman | HF | 0.12% | 0.12% | 0.18% | 0.001 | 2.48 | 0.15 | |
| 39 | davlan-mbert | HF | 0.10% | 0.27% | 0.82% | 0.001 | 2.51 | 0.15 | |
| 40 | davlan-xlmr | HF | 0.00% | 0.02% | 0.55% | 0.000 | 2.26 | 0.14 | |
| 41 | spacy-alrosait | SPACY | 0.00% | 0.00% | 0.01% | 0.000 | 0.19 | — |
How to read these numbers. Completely hidden means the whole sensitive value was masked. Found at least partly can still leave text visible. Extra text masked is measured on unlabeled rows and is not a verified false-positive rate.
Benchmark size
What was tested
These counts describe the test data; they do not show which detector is best.
Detector setups tested10362 catalog records · variants included
Language groups2English · Russian
Text rows tested6,2452,591 saved prediction runs overall
Labeled sensitive items4,128Values expanded to word boundaries