Skip to content
Crypto.world

Research

EvoPoC: Automated Exploit Synthesis for DeFi Smart Contracts via Hierarchical Knowledge Graphs

Automated tools can flag suspicious code in DeFi smart contracts, but a flagged weakness is not the same as a proven, exploitable bug — auditors need a working exploit to judge severity and…

EvoPoC: Automated Exploit Synthesis for DeFi Smart Contracts via Hierarchical Knowledge Graphs
arXiv
2605.02868
Published
2026-05-04
Authors
Ruichao Liang, Jing Chen, Xianglong Li, Huangpeng Gu, Yebo Feng, Yue Xue, Cong Wu, Yang Liu

AI summary

Overview

Research area: Blockchain and smart contract security — specifically automated vulnerability detection and proof-of-concept (PoC) exploit generation for Decentralized Finance (DeFi) protocols, combining large language models (LLMs) with knowledge graphs and formal constraint solving.

Technical level: Advanced. The paper assumes familiarity with DeFi mechanics (AMMs, flash loans, fee-on-transfer tokens), LLM agent design, retrieval-augmented generation, SMT solving, and smart contract testing frameworks such as Foundry.

Scope: The paper presents EvoPoC, a knowledge-driven agentic system that detects DeFi smart contract vulnerabilities and synthesizes executable exploit PoCs, evaluated on 88 real-world attacks and 72 audited projects (2,573 contracts).

What This Paper Is About

Automated tools can flag suspicious code in DeFi smart contracts, but a flagged weakness is not the same as a proven, exploitable bug — auditors need a working exploit to judge severity and prioritize fixes, and writing those exploits by hand is slow and needs deep expertise. Simply prompting an LLM to write an exploit does not work well, because turning a localized code anomaly into a multi-contract, stateful attack requires reasoning about protocol economics, external dependencies, and execution order. EvoPoC addresses this by treating exploit synthesis as a structured reasoning problem over an organized body of DeFi security knowledge, rather than as a code-generation task.

Key Contributions

  1. A diagnosis of why exploit synthesis is harder than detection. The authors identify three root causes: (i) the semantic distance between spotting a localized flaw and composing a full PoC; (ii) LLM hallucination and stale knowledge across long reasoning chains; and (iii) the lack of executable validity guarantees for LLM-generated PoCs.

  2. A Hierarchical Knowledge Graph (HKG). DeFi security knowledge is encoded in three layers — contract semantics (protocol, access control, economic model, dependency), failure modes (failure pattern, condition, impact, root cause, invariant violation), and exploit primitives (skeleton, primitive, example) — with typed nodes and edges that act as reusable reasoning anchors.

  3. An evolving agentic memory. A long-term memory built from audit reports, incident disclosures, and technical blogs (with explicit inclusion criteria requiring at least two of: available vulnerable source code, detailed analysis, explicit exploitation steps) is combined with task-specific working memory instantiated per contract, keeping reasoning grounded and current.

  4. A two-stage feasibility validation framework and full system. Exploit-path reachability is checked with SMT solving (Z3) over extracted require/assert/branch predicates, and profit realizability is checked via asset-level state simulation tracking attacker balances and price-relevant state; only candidates passing both reach Foundry execution. The system is implemented in roughly 12K lines of Python using LangChain, ANTLR, Neo4j, and Faiss.

Main Findings

  • Detection performance: On dataset D1 (72 Code4rena-audited projects, 2,573 contracts, including 41 projects with 48 verified vulnerability types and 31 vulnerability-free projects), EvoPoC achieves 98% recall and an F1-score of 0.9 (reported as 90% in the introduction), outperforming the LLM-based scanner GPTScan.

  • Exploit success: EvoPoC reaches a 96.6% exploit success rate (ESR) and reproduces 85 historical real-world exploits, recovering over $116.2M in revenue.

  • Against fuzzers: Compared with the state-of-the-art fuzzers Verite and ItyFuzz, the paper reports gains of up to 5× in ESR and 300× in recoverable value.

  • Against the LLM exploit generator: Compared with A1, the paper reports 2× higher ESR and 8.5× higher revenue.

  • 0-day discovery: In bug bounty evaluation on the Secure3 (D3) projects, EvoPoC discovered 21 0-day vulnerabilities, of which 16 were acknowledged or patched, helping secure over $70.6M and earning $2,900 in bounties. (The abstract states 16 confirmed 0-day vulnerabilities with the same $70.6M protected and $2,900 bounty figures.)

  • Structured knowledge beats plain chain-of-thought prompting: On the motivating fee-on-transfer price-manipulation example, a CoT-only prompt with GPT-5 identified the vulnerability 5/5 times but produced a correct root cause only 3/5 times, an exploit strategy 0/5, and an executable PoC 0/5. The knowledge-guided variant achieved 5/5, 5/5, 5/5, and 4/5 respectively, while needing fewer iterations (1, 1.4, 2, 2 versus 1.8, 4, 5, 5).

  • Backbone model coverage: Five LLMs were used — GPT-5.2, GPT-5, GPT-4o, GPT-o3, and GPT-3.5-turbo — at temperature 0.2 via public API endpoints. Per-model detection breakdowns are presented in the paper's Table II, which is truncated in the provided text and therefore not reported here.

  • Context: The introduction cites an estimated $3.6 billion lost to smart contract flaws in 2025 alone.

Methodology in Plain English

EvoPoC works in three phases.

Phase 1 — Build the knowledge base. The team defines an ontology: a fixed set of node types and allowed relationships describing DeFi protocols, how they fail, and how attackers exploit them. An LLM reads real vulnerability intelligence (audit reports, post-mortem disclosures, analysis blogs) and extracts structured nodes and edges into this schema. New cases are merged into a single global graph: a type-filtered lookup narrows candidates, a Faiss approximate-nearest-neighbor search over node-description embeddings finds the top-k semantic matches, and equivalent nodes are merged while near-duplicates at different granularities are kept as variants. Because new intelligence goes through the same pipeline, the graph keeps growing.

Phase 2 — Reason over the graph for a target contract. Given a contract, the system filters out non-Solidity files, tests, and trusted libraries, then summarizes the contract semantically. It retrieves the closest protocol-level nodes as starting points and expands hop by hop within the contract-semantics layer, using chain-of-thought reasoning to judge relevance at each step. To move down a layer, it first tries sparse retrieval — checking whether several selected semantic nodes converge on the same failure pattern via existing cross-layer edges — and falls back to dense embedding similarity when that yields nothing. The same procedure repeats from failure modes to exploit primitives. Everything retained is assembled into a task-specific subgraph that acts as working memory for the exploit-writing step.

Phase 3 — Write and check the exploit. The LLM first drafts a high-level plan split into preparation, exploitation, and extraction, with each transaction described by its target contract, entry function, parameters, and intended effect. A reachability check walks the call graph, tracks symbolically how parameters propagate across functions and contracts, encodes branch conditions as Z3 formulas, and discards the candidate on UNSAT. Survivors are turned into a Foundry test script, then passed through an asset-level simulator that tracks attacker balances and AMM/oracle price state under optimistic assumptions (zero slippage, successful branches); only scripts with positive net wealth change are actually executed. The design deliberately avoids heavyweight symbolic verification — the goal is cheap pruning that returns a reason for failure the LLM can act on, since a bare Foundry revert gives little diagnostic signal.

Why This Matters

Impact on research. The paper reframes LLM-based exploit generation as knowledge-grounded structured reasoning rather than code completion, and shows measurable degradation of plain prompting as reasoning depth increases. It also offers a reusable pattern — a typed domain ontology as LLM memory, plus cheap formal checks as a hallucination filter — that extends beyond DeFi to other security domains where detection and exploitation are far apart.

Real-world applications:

  • Audit triage: turning a long list of static-analysis alerts into ranked, demonstrated exploits so auditors focus on what is actually reachable and profitable.
  • Bug bounty work: automatically producing the PoC that bounty programs require before a report is accepted.
  • Protocol pre-deployment review: teams testing their own contracts in local forks before shipping.
  • Post-incident analysis: replaying historical attacks to confirm which disclosed vulnerabilities are reproducible on a given fork.

Industry relevance. DeFi losses are large and recurring (the paper cites $3.6 billion in 2025), and the gap between an unverified alert and a confirmed exploit is exactly where remediation slows down. Reproducing 85 historical exploits and surfacing patched 0-days with bounty payouts suggests the approach is close enough to practice to matter to auditors, bounty hunters, and protocol teams.

Future Directions

  • Scaling and maintaining the knowledge graph. The paper stresses that novel attack patterns outpace model training data; how quickly and how accurately the fusion pipeline absorbs genuinely new primitives — and whether the LLM-based equivalence judgments introduce silent errors — remains an open question.
  • Strengthening validity guarantees. Reachability checking deliberately treats unrelated external contracts with conservative defaults, admitting some candidates rather than risking false rejection, and profitability simulation assumes idealized conditions (zero slippage, successful branches). Tightening these without losing scalability is a natural next step, as is extending the checks to multi-transaction, cross-block attacks.
  • Broadening beyond DeFi and Solidity. The ontology is DeFi-specific; whether the same three-layer structure transfers to other contract domains, other chains, or non-financial smart contract risks is untested.
  • Cost and latency in practice. RQ1 in the paper asks about runtime and token consumption, but those figures fall in the truncated portion of the text and are not reported here; how the multi-hop retrieval, SMT checks, and simulation budget trade off against throughput for large audit portfolios is an open practical question.

Target Audience

Security researchers and graduate students working on LLM agents, program analysis, or smart contract security; auditors and bug bounty hunters who need reproducible exploits rather than alerts; and DeFi protocol engineers or security tooling vendors evaluating whether automated exploit synthesis is mature enough to fold into their review pipeline. Readers without blockchain background will need to acquire DeFi mechanics (AMMs, flash loans, fee-on-transfer behavior) and Foundry basics first.

Authors’ abstract

Smart contract vulnerabilities in Decentralized Finance caused over billions of dollars losses every year, yet the security community faces a critical bottleneck: identifying a vulnerability is not the same as proving it is exploitable. Manual PoC construction is prohibitively labor-intensive, leaving most disclosed vulnerabilities unverified and protocols exposed long before mitigation is applied. In this paper, we propose \sys, a knowledge-driven agentic system for…

The full abstract on arXiv
Read the original paper