Research
Towards Automating Blockchain Consensus Verification with IsabeLLM
Consensus protocols are what let blockchain nodes agree on state in an adversarial environment, so mistakes in their design or implementation can be exploited — the paper cites 51% attacks on…

- arXiv
- 2601.07654
- Published
- 2026-01-12
- Authors
- Elliot Jones, William Knottenbelt
AI summary
Overview
Research area: Formal verification of blockchain consensus protocols, specifically AI-assisted theorem proving using the Isabelle proof assistant.
Technical level: Intermediate. The paper assumes familiarity with proof assistants, Higher-Order Logic, and blockchain consensus concepts, though its narrative is readable for those with a general computer science background.
Scope: The paper presents IsabeLLM, a tool connecting Isabelle to any LLM API, and evaluates it by building and verifying a novel n-ary tree model of Bitcoin's Proof of Work consensus using the DeepSeek R1 API.
What This Paper Is About
Consensus protocols are what let blockchain nodes agree on state in an adversarial environment, so mistakes in their design or implementation can be exploited — the paper cites 51% attacks on Ethereum Classic, Bitcoin Gold, and Vertcoin totalling over $30 million in losses. Formal verification could prevent such failures, but it demands high effort and expertise, so it is often skipped, and patching a deployed consensus protocol typically requires a disruptive hard fork. The authors build IsabeLLM to reduce that barrier, pairing Isabelle's automation tools with an LLM, and demonstrate it by proving correctness of a new mechanised model of Bitcoin's Proof of Work consensus.
Key Contributions
- The IsabeLLM tool itself — a general-purpose interface between Isabelle and any LLM API, usable for theorem proving in any domain, with its architecture described in Section 5.1 and implementation in Section 5.2.
- A novel mechanised model of Bitcoin's Proof of Work consensus in Isabelle, extending prior work from a binary tree to an n-ary tree so that an arbitrary number of forks per block can be represented, with correctness proven using IsabeLLM and the DeepSeek R1 API.
- An empirical performance analysis of IsabeLLM across 16 lemmas, reporting success rate, iteration counts, and the pain points that emerged (Section 6).
- A comparison of proof effort between the binary tree and n-ary tree models, quantifying how generalisation increases proof complexity.
Main Findings
- Proof generation succeeded on every non-trivial lemma: IsabeLLM produced correct proofs for each of the non-trivial lemmas in the verification, though the number of successful attempts varied by lemma.
- Several lemmas were solved 10 out of 10 times:
subtree_height,height_mono,weaken_distance,weaken_depth, andconsensuseach succeeded in all 10 attempts, andcheck_add (mining)also reached 10 successes with an average of 1 iteration and a listed proof length of 1 line. - Harder, longer proofs had lower success rates:
foldr_max_eqandsub_brancheach succeeded 5 out of 10 times,common_prefixandheight_add (mining)6 times, andsub_longest7 times. - Two lemmas required Isabelle2025's Sledgehammer:
branch_height(3 successes) andbounded_check(7 successes) are marked as completed using Sledgehammer from Isabelle2025, since Sledgehammer in the Isabelle2022 setup struggled to close intermediate steps even when the LLM's high-level proof structure was sound. - Failure mode is fixation on a single step: When a proof step failed and Sledgehammer found nothing, the LLM often resubmitted a slightly modified version of the same step, creating a loop of tweaking without progress. IsabeLLM was most successful when the LLM decomposed the step into smaller parts that Sledgehammer could solve.
- The n-ary generalisation is substantially harder to prove: In Table 1, the binary tree model totals 175+193 lines of proof while the n-ary tree totals 395; the paper states the n-ary model has more than double the original lines of proof compared to the binary tree model.
- API limitations were a practical constraint: The free OpenRouter API for DeepSeek R1 was slow at times and occasionally returned empty output, requiring IsabeLLM to detect and repeat the iteration.
Methodology in Plain English
The workflow is iterative and builds on Isabelle's own error reporting. The user places an Isabelle theory file and a ROOT file in a working directory and starts IsabeLLM. The tool first asks the Isabelle server to build the theory file; if everything is already proven, it stops. Otherwise it reads the build errors to identify unproven statements and extracts the one it wants to work on.
That lemma plus the surrounding theory context is sent to the LLM through its API, and the returned proof is injected back into the file for another build attempt. If the build still fails, IsabeLLM hands the failing lines to Isabelle's Sledgehammer tool, then returns the remaining errors and the updated proof state to the LLM. Steps four through six repeat until the file builds or a preset maximum number of iterations is reached.
The tool is written mostly in Scala so it can use the Scala-Isabelle library, which lets a Scala application control an Isabelle process; some Python uses the openai library to reach the LLM API. Sledgehammer is given a 60-second timeout and the default provers. The prompt strategy is two-part: an initialisation prompt containing the theory file so far and the target lemma, then error prompts containing only the current proof state and the specific error. Chat history is reset after each successful lemma to keep context small.
For evaluation, each of the 16 lemmas in Table 1 was attempted 10 times with a maximum of 5 iterations per attempt, excluding cases where the LLM returned an empty response. An attempt counts as a failure if it exceeds the iteration limit or exits prematurely, often due to syntax issues.
Why This Matters
Impact on research: The paper connects two lines of work that have barely met — AI for theorem proving, which has largely targeted mathematical statements, and formal verification of blockchain systems, where the authors describe the literature on AI-assisted verification as sparse. It also contributes a reusable mechanised model of PoW consensus with a realistic n-ary fork structure.
Real-world applications:
- Verifying consensus protocols before deployment, avoiding the hard forks that patching would otherwise require (the paper points to the Ethereum/Ethereum Classic split as an example).
- Auditing smart contracts, which are usually immutable once deployed, so correctness-by-construction reduces the need for costly post-deployment fixes.
- Securing bridging protocols for cross-chain data transfer, a category the paper links to Poly Network, Wormhole Bridge, Binance Smart Chain, and Qubit Finance, with combined losses over $1.5 billion.
- Lowering the expertise barrier so developers without deep proof-assistant training can attempt mechanised correctness proofs.
Industry relevance: Bitcoin's scale — roughly 1.2 billion recorded transactions and a market capitalisation of around $1.8 trillion — means consensus bugs carry enormous financial exposure, giving institutions a direct incentive to adopt cheaper verification workflows.
Future Directions
- Parallel proof-tree search: Query the LLM in parallel and branch the proof in different directions for each candidate proof, the standard approach in AI for theorem proving (the paper cites DeepSeek-Prover and LISA), which would help IsabeLLM avoid getting stuck on one step. Extending this to multiple different LLMs could diversify proof strategies.
- Compatibility with newer Isabelle releases: IsabeLLM runs on Isabelle2022 for Scala-Isabelle compatibility, missing the improved Sledgehammer in later versions. Restoring local Sledgehammer would also let nitpick detect impossible proofs before provers are invoked, and would allow caching of repeated successful steps to avoid recomputation.
- Scaling beyond single theory files: Applying IsabeLLM to more complex proofs outside the blockchain domain that span multiple theory files, and pairing it with existing frameworks such as Isabelle/Solidity so users need only specify the invariant rather than building the whole model.
- Reducing hallucination and repetition: Fine-tuning on proof corpora such as the Archive of Formal Proofs, building bespoke Isabelle-specific language models in the style of LISA, or applying Retrieval-Augmented Generation and Static Prompt Templating to make the tool less dependent on LLM quality.
Target Audience
Researchers and practitioners in formal verification and proof engineering who want to reduce the manual effort of Isabelle proofs; blockchain protocol designers and auditors interested in mechanised consensus verification; and AI-for-theorem-proving researchers looking for a concrete program-verification case study with reported success rates and failure modes. Readers seeking a ready-made end-to-end verifier should note the stated limitation that IsabeLLM automates proofs of statements but not the generation of the statements themselves — functions, locales, sets, and the target lemma must be specified by the user first.
Authors’ abstract
Consensus protocols are crucial for a blockchain system as they are what allow agreement between the system's nodes in a potentially adversarial environment. For this reason, it is paramount to ensure their correct design and implementation to prevent such adversaries from carrying out malicious behaviour. Formal verification allows us to ensure the correctness of such protocols, but requires high levels of effort and expertise to carry out and thus is often omitted in the…
The full abstract on arXiv