# An Effective and Cost-Efficient Agentic Framework for Ethereum Smart Contract Auditing

Crypto.world

> Smart contract audits are expensive and unscalable when done manually, static analyzers miss protocol-specific logic bugs and flood auditors with false positives, and existing LLM-based auditors…

**arXiv**: 2601.17833

**Published**: 2026-01-25

**Authors**: Xiaohui Hu, Wun Yu Chan, Yuejie Shi, Qumeng Sun, Wei-Cheng Wang, Chiachih Wu, Haoyu Wang, Ningyu He

## AI summary

## Overview

**Research area:** Smart contract security — specifically LLM-driven, agentic vulnerability auditing for Ethereum/DeFi protocols, with emphasis on detecting business logic vulnerabilities before deployment.

**Technical level:** Advanced. The paper assumes familiarity with static analysis (Slither), graph clustering (Louvain, betweenness centrality, PageRank), LLM agent architectures and the Model Context Protocol (MCP), and SMT-based symbolic constraint solving (Z3).

**Scope (one sentence):** The paper presents Heimdallr, a three-stage LLM agent pipeline (code partitioning, "Plan-Remind-Solve" auditing, and adversarial false-positive verification) that aims to deliver high-precision smart contract auditing at low cost using open-source models rather than proprietary ones.

## What This Paper Is About

Smart contract audits are expensive and unscalable when done manually, static analyzers miss protocol-specific logic bugs and flood auditors with false positives, and existing LLM-based auditors hallucinate findings or depend on costly proprietary models. The authors build Heimdallr to keep business logic intact while shrinking what the model must read, to reason about complex and even chained exploits, and to filter out implausible findings before reporting them. The goal is an audit pipeline that is simultaneously more accurate and dramatically cheaper than state-of-the-art industrial and academic baselines.

## Key Contributions

1. **A novel auditing methodology.** Heimdallr combines graph-theoretic code profiling with neuro-symbolic reasoning: a "Plan-Remind-Solve" workflow for detecting complex logic vulnerabilities, paired with an adversarial verification module intended to suppress false positives.
2. **A production-ready auditing service.** The authors release Heimdallr as a standardized API service for integration into CI/CD pipelines, and support privacy-preserving local deployment on open-source models for auditing proprietary code in air-gapped environments.
3. **A comprehensive empirical evaluation.** Heimdallr is benchmarked against four state-of-the-art baselines, reporting an F1 score of 0.62 on real-world datasets, described by the authors as orders of magnitude higher than existing academic prototypes; the paper also claims the first rigorous analysis of the precision-recall trade-off in LLM-based auditing, motivating decoupled verification.
4. **High-fidelity vulnerability benchmarks.** Three distinct datasets are constructed and released, covering high-value zero-day exploits, standardized benchmarks, and crowdsourced contest findings.

Note on availability: the authors explicitly state they will **not** open-source the audit engine to prevent misuse; access is available by request via the API service (contacts: Xiaohui Hu, Chiachih Wu, Ningyu He).

## Main Findings

- **Real-world exploit reconstruction:** Heimdallr successfully reconstructed 17 out of 20 real-world attacks occurring after June 2025, corresponding to total losses of $384M.
- **Zero-day discovery:** The framework uncovered 4 confirmed zero-day vulnerabilities in live protocols, which the paper states safeguarded $400M in TVL.
- **Efficiency versus baselines:** Compared to state-of-the-art baselines including both official industrial tools and academic tools, Heimdallr at most reduces analysis time by 97.59% and financial costs by 98.77%, while boosting detection precision by over 93.66%.
- **Audit contests:** When applied to auditing contests, Heimdallr achieves a 92.45% detection rate at a cost of $2.31 per 10K LOC, and ranked 19th out of 548 participating human auditors.
- **Prior LLM tools are impractical:** Applying three existing LLM-based tools to 80 real-world protocols, the authors found either negligible detection capability (F1 Score < 0.02) or prohibitive noise levels (FPR > 97%).
- **Prior analyzers miss logic bugs:** An analysis cited by the paper of 127 real-world attacks totaling $2.3 billion in losses found that existing tools detect vulnerabilities in only 25% of cases, accounting for merely 12% of total damages ($271 million); 38.1% of auditors reported existing tools were unhelpful.
- **The cost-accuracy dilemma:** Comprehensive analysis reportedly demands high-capability models such as GPT-5, with a single audit project frequently costing $20–$50 in the authors' practice; substituting a model like GPT-oss-120B caused detection recall to plummet by 60% to 80%. Heimdallr's design targets high performance on GPT-oss-120B without proprietary systems.

Items **not reported** in the supplied text: the specific names and sizes of the three released datasets, the specific names of the four baselines, per-baseline result tables, the internal details of the False Positive Filtration stage (Section 4.3 is truncated), and the implementation/evaluation sections (§5–§6).

## Methodology in Plain English

The pipeline has three coupled stages.

**Stage 1 — Splitting the code without losing meaning (Contextual Profiling).** A whole project is too large to feed to a model, but splitting by file breaks logic that spans contracts. The authors use Slither to build a directed graph whose nodes are individual functions. Edges capture direct calls (weight 1.0) and state-variable data dependencies, where one function writes a state variable another reads (weight 0.8, empirically tuned). They run the Louvain algorithm to find densely connected communities, then map each community back to the contracts containing its functions, forming batches; batches may overlap to cover shared libraries or core utilities. Inside each batch, contracts are ranked by an importance score: the average over their functions of α times betweenness centrality plus β times PageRank, so "anchor contracts" get more of the model's attention. Finally an LLM refines the batches semantically — merging batches that carry the same or related business tags (for example, all parts of a "Swap" flow) and, when a batch exceeds the token limit, repeatedly removing the most common non-customized contracts (an ERC-20 template, say) until it fits.

**Stage 2 — Auditing with a "Plan-Remind-Solve" agent (Model-Agnostic Auditing).** The audit is formalized as producing raw vulnerability hypotheses from the batches plus a curated knowledge base built from public audit checklists (e.g., Solodit) and historical attack reports from Web3 security organizations (e.g., DeFiHackLabs, BlockSec), where each vulnerability category file stores a pattern, a concrete exploit instance, and an expert reasoning trace.

- *Plan:* the LLM scans each contract's functions asking whether they could be vulnerable; for flagged functions, callers and callees are extracted from the graph into a task, relevant knowledge files are attached, and tasks are queued, sorted first by contract importance score and then by the LLM's judgment of potential severity.
- *Remind:* relevant exploit instances and reasoning traces are pulled in as context, and a custom MCP server uses a `check_protocol_lineage` tool to detect whether a contract is a derivative of known DeFi primitives (e.g., a Compound fork); if so, a live search for recent incidents on that protocol is merged into the context.
- *Solve:* beyond a baseline LLM audit, the agent performs adversarial state assumption analysis under three profile classes — environment tampering (manipulating `block.timestamp` or `block.number`), interaction hijacking (malicious calldata, forced return values), and resource infinity (flash-loan-scale capital) — and targeted symbolic constraint solving, where the LLM transpiles numerically sensitive Solidity into Z3 constraints (mapping `uint256` to SMT bit-vectors), adds real-world sanity constraints such as no negative balances, and solves for concrete values. A final iterative combination step addresses "cognitive tunneling" by chaining hypotheses: if the post-condition of one finding satisfies the pre-condition of another, they are merged into a composite exploit scenario, repeating until no new chains appear.

**Stage 3 — Filtering hallucinations (False Positive Filtration).** A dedicated verifier acts as an adversarial quality gate, checking each hypothesis for feasibility against global safeguards and realistic threat models, and prunes the raw set down to a final report. The supplied text cuts off at the start of this stage, so the internal mechanics and any reported ablation of the verifier are not available here.

Each hypothesis is stored as a structured triple of entry point (function and line number), constraint set (preconditions for exploitation), and reasoning path, so that vague suspicions become falsifiable claims that the verifier can attack.

## Why This Matters

**Impact on research.** The paper reframes LLM-based auditing as a precision problem rather than a raw-detection problem, arguing that decoupled verification is necessary because detection and false-positive suppression place conflicting demands on a single model. It also targets a concrete, widely reported gap: static analyzers cover only a small fraction of real exploit losses because business logic bugs are context-dependent, and prior LLM tools were shown here to be either blind (F1 < 0.02) or unusably noisy (FPR > 97%). Its claim of strong results on GPT-oss-120B rather than GPT-5 pushes back on the assumption that audit quality requires the most expensive proprietary models.

**Real-world applications.**

- Pre-deployment CI/CD gating: contracts audited automatically on each commit via the API service.
- Privacy-sensitive or air-gapped auditing: local deployment with open-source models for proprietary codebases that cannot be sent to third-party servers.
- Continuous monitoring of live protocols: the MCP lineage check and live incident search let the agent incorporate newly reported attacks against forked or derivative protocols.
- Triage support for human auditors: contract ranking, function-level entry points, and constraint sets can prioritize review rather than replace it.

**Industry relevance.** The economics are the headline: manual audits are cited at $2,500 for roughly 1,000 lines of code, and $70,000–$150,000+ for a medium DeFi protocol, while existing cloud-based LLM agents cost $15–$50 per 10K-line project. Against a backdrop of over 700 security incidents and roughly $3.35 billion in losses (a 66.64% increase year over year) protecting $123.6 billion in on-chain assets, a claimed $2.31 per 10K LOC at a 92.45% detection rate — and a rank of 19th out of 548 human auditors in contests — is directly aimed at audit firms, DeFi teams, and security tooling vendors.

## Future Directions

- **Verification as a first-class research object.** The paper positions decoupled verification as essential, but the details of the adversarial feasibility checks are the least exposed part of the design in the supplied text; quantifying how much precision each verification sub-step buys is the natural next experiment.
- **Keeping the knowledge base current.** The static knowledge base is acknowledged as potentially missing emerging vulnerabilities, which the MCP live-search path only partly compensates for; how well the system generalizes to genuinely novel attack classes, as opposed to variants of known ones, remains open.
- **Independent reproduction.** Because the audit engine is deliberately closed and access is by request, the released datasets and benchmarks carry most of the reproducibility burden; third-party validation on the three released datasets is the obvious next step.
- **Cost frontier beyond GPT-oss-120B.** The paper demonstrates the approach without proprietary systems, but where the accuracy floor lies — how much smaller or more quantized a local model can get before the reported F1 of 0.62 degrades — is not reported in the available text.

## Target Audience

Smart contract security auditors and audit firms evaluating automation; DeFi protocol engineers and DevSecOps teams deciding how to gate deployments; LLM-agent researchers interested in tool-augmented reasoning, neuro-symbolic hybrids, and hallucination suppression; and empirical software-security researchers who want the released exploit and contest datasets for comparative benchmarking. Readers need comfort with static analysis, graph algorithms, and LLM pipelines; newcomers will find the motivation and cost comparisons accessible but the methodology dense.

## Authors’ abstract

Smart contract security is paramount, but identifying intricate business logic vulnerabilities remains a persistent challenge because existing solutions consistently fall short: manual auditing is unscalable, static analysis tools are plagued by false positives, and fuzzers struggle to navigate deep logic states within complex systems. Even emerging AI-based methods suffer from hallucinations, context constraints, and a heavy reliance on expensive, proprietary Large Language…

[The full abstract on arXiv](https://arxiv.org/abs/2601.17833)

[Read the original paper](https://arxiv.org/abs/2601.17833)

---

Canonical: https://crypto.world/paper/an-effective-and-cost-efficient-agentic-framework-for-ethereum-smart-2601.17833
