Research
ZKP Security Tools and Verification: Coverage, Effectiveness, Adoption, and Challenges
ZKP systems secure blockchains handling billions of dollars and identity applications handling sensitive personal data, but subtle bugs in circuits or proof systems can let an attacker forge money or…

- arXiv
- 2607.23752
- Published
- 2026-07-26
- Authors
- Arman Kolozyan, Tom Sorger, Alexander Hicks, Stefanos Chaliasos
AI summary
Overview
Research area: Applied cryptography and security engineering — specifically the tooling ecosystem around zero-knowledge proofs (ZKPs), covering static analysis, fuzzing, SMT-based verification, formal verification in proof assistants, and practitioner workflows.
Technical level: Intermediate. The tool-landscape and empirical evaluation sections are accessible to anyone who knows what a circuit constraint is; the formal verification section (Section VI) assumes familiarity with proof assistants (Lean, Rocq, ACL2, EasyCrypt), trusted computing bases, and zkVM architecture, and is closer to Advanced.
Scope (one sentence): A mixed-methods study that systematizes ZKP security tools, empirically measures how well six of them find 70 real-world bugs, systematically analyzes formal verification efforts across the zkVM stack, and surveys 48 practitioners about adoption and gaps.
What This Paper Is About
ZKP systems secure blockchains handling billions of dollars and identity applications handling sensitive personal data, but subtle bugs in circuits or proof systems can let an attacker forge money or false proofs. Many bug-finding and formal verification tools have been proposed in response, yet nobody had measured whether they actually work on real code or whether anyone uses them. This paper closes that gap by combining a tool systematization, a hands-on evaluation, a formal-verification survey of the literature, and a practitioner survey, structured around five research questions (RQ1–RQ5).
Key Contributions
-
A systematization of the ZKP security tool landscape (RQ1). Twelve tools are compared in Table II (Circomspect, CCC-Check, Pilspector, Korrekt, zkFuzz, Zequal, ConsCS, Picus, Civer, Ecne, ZKAP, AC4) by analysis target, technique (static analysis, dynamic analysis, formal verification, or combinations), vulnerability coverage, open-source status, and automation level. The finding is that tools focus almost exclusively on Circom circuits and on nondeterminism/underconstraint bugs.
-
An empirical evaluation of six Circom tools on a real-world bug dataset (RQ2). The authors extend the Chaliasos et al. dataset of real-world circuit vulnerabilities from 29 to 70 bugs, and run each tool in two modes: an isolated wrapper around the vulnerable code, and the project mode of the original codebase.
-
The first systematic analysis of formal verification efforts for ZKP systems (RQ3). Table V catalogues efforts across the zkVM stack by verification goal (constraint correctness, proof-system soundness, verifier correctness, witness–constraint consistency), model-construction method (extraction, translation via compiler IR, hand-written model, correctness by construction), proving technique, and tooling.
-
A practitioner survey on adoption and perceived gaps (RQ4, RQ5). 48 respondents out of 200 contacted (a 24% response rate), split into 36 developers/researchers and 12 auditors, covering tool usage, formal verification workflows, and LLM-assisted security practice.
-
A public artifact at
https://github.com/t-sorger/zkp-security-toolscontaining the extended bug dataset, tool selection process, evaluation harness and results, and anonymized raw survey data.
Main Findings
-
Tool coverage is narrow (RQ1). Most tools target Circom at either source level (e.g., Circomspect) or constraint level (e.g., Picus on R1CS); newer DSLs and zkVMs have limited support. Tools concentrate on nondeterminism, with weaker or specification-dependent coverage of missing range checks, division by zero, circuit logic assumptions, other mismatches, and overconstrainedness. Semantic mismatches and cryptographic issues are largely unsupported and omitted from the comparison table entirely.
-
Detection on isolated bugs: 45.7%. At least one tool detects 32 of 70 bugs, producing 83 true positives across all executions. This masks overlap: 38 bugs are missed by every tool, and 20 bugs are detected by at least two.
-
Detection on full codebases: 19.6%. At least one tool detects only 11 of the 56 project-mode-compilable bugs, with 14 true positives in total. All 70 bugs compile in isolated mode; the remaining projects fail to compile because of stale code or deprecated dependencies.
-
Isolated benchmarks overestimate real effectiveness. Among the 56 bugs that compile in both modes, 31 are missed in both modes, 6 are detected in both, and 14 are detected only in isolated mode.
-
Detection is concentrated by root cause (Table IV). In isolated mode: assigned-but-unconstrained 6/6 (100.0%), missing input constraints 10/22 (45.5%), logic-to-constraint translation 9/18 (50.0%), unsafe circuit reuse 4/10 (40.0%), arithmetic field issues 2/4 (50.0%), specification misimplementation 1/3 (33.3%), circuit design issue 0/6 (0.0%), other programming errors 0/1 (0.0%). In project mode every class weakens: assigned-but-unconstrained 2/5 (40.0%), missing input constraints 3/18 (16.7%), logic-to-constraint translation 3/13 (23.1%), unsafe circuit reuse 2/7 (28.6%), arithmetic field issues 1/4 (25.0%), circuit design issue 0/6 (0.0%), specification misimplementation 0/2 (0.0%), other programming errors 0/1 (0.0%).
-
Tools are operationally immature. Civer detects 20 true positives in isolated mode but also fails with 26 errors and 14 timeouts. Most timeouts stem from the SMT queries the tools perform, and errors and timeouts become more common in project mode. Circomspect never times out but detects only one true positive. Ecne and Circomspect were observed to be substantially noisier (more false positives) than the remaining tools, though false-positive rates were not systematically measured.
-
Formal verification is isolated and lopsided (RQ3). No zkVM has end-to-end formal verification. Constraint correctness is the most mature goal; the witness generator, lookup/permutation arguments, proof-system backend, and on-chain verifier remain largely unverified. zkVM verification is converging on Lean and the Sail RISC-V specification, with Rocq, ACL2, and EasyCrypt also used. OpenVM's constraint-correctness proofs, for example, assume the soundness of the lookup/bus argument tying its chips together.
-
Model construction determines the trusted computing base. Extraction (as in CertiPlonk, OpenVM, and the original SP1 Hypercube proofs) keeps the model close to what the prover evaluates but trusts the extractor and loses code structure. Translation via a compiler IR (e.g., LLZK lowering to Picus in RISC Zero's CI/CD pipeline, or rocq-of-rust) makes the compiler pipeline a major trust assumption, and automated Rust-to-proof-assistant tools translate only a fragment of Rust — SIMD intrinsics,
unsafe, and inline assembly are unsupported. Hand-written models (CertiK's Rocq model of zkWasm, Jolt's ACL2 model, StarkWare's Lean Cairo model) are flexible but need separate validation, e.g., Jolt exhaustively compared subtable outputs against Rust where feasible (at most 2^16 entries) and random-tested otherwise. Correctness by construction (Clean, in Lean) removes the model–implementation gap but requires rewriting the artifact in the proof assistant. -
Complexity gets shifted rather than removed. Lookup arguments are advertised as simplifying zkVMs and constraints, but end-to-end verification must then also cover the lookup argument's specification, its underlying cryptography, and the well-formedness of its tables.
-
LLMs are already mainstream in practice (RQ4/RQ5). Workflows remain human-led, but interactive LLMs are used by 85% of developers and 83% of auditors. Advanced tools are seen as useful but costly to integrate, often requiring project-specific specifications and manual validation. Practitioners view underconstrained bugs as the most mature target for current tooling, identify major gaps around semantic errors, Fiat–Shamir issues, and integrations, and prioritize formal verification, LLMs, and tools with clear reports and guarantees.
Methodology in Plain English
Landscape and literature (RQ1, RQ3). The authors assembled two corpora from academic papers, GitHub repositories, and blog posts/project documentation: one of tools that analyze ZKP circuits for security bugs, and one of formal verification efforts targeting ZKP systems, with a focus on zkVMs. Two authors independently classified each artifact — tools by target, technique, supported vulnerability classes (mapped onto existing ZKP bug taxonomies), source availability, and automation level; verification efforts by goal, model-construction method, proving technique, scope, and trusted computing base. Disagreements were resolved by discussion, with a third author reviewing unresolved cases. Closed-source work was included only where public material allowed classification.
Tool evaluation (RQ2). They restricted the evaluation to Circom, since it is both the most-targeted and one of the most mature and widely deployed DSLs, and selected all open-source tools that run non-interactively, using only modes that require no manual annotations. Rather than tool-specific benchmarks, they used real vulnerabilities from audit reports and public disclosures, extending a prior dataset from 29 to 70 bugs. Each bug can be run two ways: wrapped in a minimal harness around the vulnerable code (isolated) or inside the original project (project). A custom harness executes tools with a 600-second timeout on a VM with 8 vCPUs, 32 GB RAM, and an AMD EPYC-Milan CPU at 2.4 GHz, and labels each run detected, missed, error, or timeout; ambiguous outputs were manually inspected by two authors against the ground-truth annotation. Only true-positive detection is measured; false positives are left to future work.
Survey (RQ4, RQ5). Two questionnaire variants were drafted (one for developers/researchers, one for auditors), refined through two pilot rounds with N = 3 participants per questionnaire per round. The authors contacted 200 practitioners, mostly at research, development, and security organizations, through personal channels, and received 48 responses (24%). Multiple-choice and Likert responses are reported as distributions over respondents to each question; open-ended answers were grouped into themes by inductive coding.
Why This Matters
Impact on research. The gap between 45.7% detection on isolated bugs and 19.6% on full codebases is direct evidence that tool papers evaluated on curated, tool-specific benchmarks overstate real-world effectiveness. The formal-verification analysis gives the field a shared vocabulary — constraint correctness, proof-system soundness, verifier correctness, witness–constraint consistency, prover completeness — plus an explicit accounting of what sits in the trusted computing base, which is exactly the kind of bookkeeping that CompCert's verification required.
Real-world applications at stake:
- Private payments: the Orchard vulnerability in Zcash remained hidden for years and could have enabled counterfeiting in its largest shielded pool.
- DeFi / bridges: an Aztec Connect exploit drained roughly $2.1 million.
- Blockchain scaling via zk-rollups, where a soundness failure in a shared zkVM is a single point of failure for many independent applications.
- Identity protocols and verifiable machine learning, where forged proofs of identity or of computation undermine the privacy and integrity guarantees users were promised.
- Bug-bounty programs, which the paper notes continue to surface high-severity ZKP bugs.
Industry relevance. The evaluation speaks directly to engineering decisions: teams should use these tools because they do find critical bugs, but should treat them as focused code-fragment analyzers rather than push-button whole-project auditors, and should budget for errors, timeouts, and false positives. The observation that LLMs are used by 85% of developers and 83% of auditors, while advanced formal tools remain costly to integrate, tells tool builders where the adoption bottleneck actually is. The convergence on Lean and the Sail RISC-V specification, plus coordinated efforts like the Ethereum Foundation's verified-zkEVM project and standardization of a common RISC-V profile, suggests the ecosystem is consolidating around shared infrastructure.
Future Directions
-
End-to-end, CI-friendly zkVM verification. The paper calls for proofs that span the whole stack rather than isolated components, integrated into continuous integration, with precise and non-vacuous claims. Open problems include verifying the witness generator, the lookup/permutation arguments, the proof-system backend, and the on-chain and recursive verifiers.
-
Better SMT performance and complementary analyses. Timeouts dominate failures and stem from finite-field SMT queries; the authors point to dedicated finite-field SMT solvers and to discharging finite-field obligations in a proof assistant instead (e.g., BitModEq as a Lean tactic, reported to outperform general-purpose SMT solvers on real zkVM benchmarks).
-
Broader language and vulnerability coverage. Tools should move beyond Circom via language-agnostic analyses or support for emerging DSLs and zkVMs, and beyond nondeterminism to semantic mismatches, Fiat–Shamir issues, integration bugs, and overconstrainedness — the classes practitioners flagged as hardest to catch.
-
False-positive measurement and real-world dataset evaluation. The authors explicitly leave false-positive analysis to future work, and argue that the field needs evaluation on real-world datasets rather than narrow, tool-specific suites.
Target Audience
- ZKP and smart-contract security auditors deciding which tools to add to an audit workflow, and what to expect from them.
- Circuit and zkVM developers who want to know which bug classes current tooling actually catches on realistic codebases.
- Formal verification researchers looking for a map of what has been proven about zkVMs, with which techniques, and what remains in the trusted base.
- Tool builders and funders — the paper's "calls to action" name concrete engineering and coverage gaps worth targeting.
- Engineering managers and protocol teams assessing risk exposure and interpreting vendor or paper claims about tool effectiveness.
Authors’ abstract
Zero-knowledge proofs (ZKPs) have become a core technology for privacy and verifiable computing. They are used to secure blockchains that handle billions of dollars and identity applications dealing with sensitive personal data. However, ZKP systems are complex, and subtle implementation errors can completely break their guarantees, letting attackers forge money or false proofs of identity. Researchers and practitioners have therefore developed a growing set of bug detection…
The full abstract on arXiv