# Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization

Crypto.world

> Most smart contract vulnerabilities are entangled with project-specific business logic, so generic oracles such as "profit generation" or "reentrancy" miss them and automated tools produce high…

**arXiv**: 2603.26270

**Published**: 2026-03-27

**Authors**: Ziqiao Kong, Wanxu Xia, Chong Wang, Yue Xue, Yi Lu, Pan Li, Shaohua Li, Zong Cao, Yang Liu

## AI summary

## Overview

**Research area:** Smart contract security, specifically automated vulnerability detection for DeFi protocols using large language models (LLMs), knowledge graphs, and multi-agent workflows. Category: Cryptography and Security (arXiv:2603.26270v2 [cs.CR]).

**Technical level:** Advanced. The paper assumes familiarity with Solidity, EVM execution, Foundry testing, DeFi accounting mechanisms (e.g., ERC4626-style proportional shares), knowledge-graph schemas, and LLM agent architectures.

**Scope (one sentence):** The paper builds an auditing knowledge graph from 331 historical Code4rena contests and drives a multi-agent LLM pipeline with it to detect smart contract vulnerabilities, evaluated on 11 held-out Code4rena projects and 7 live real-world projects.

## What This Paper Is About

Most smart contract vulnerabilities are entangled with project-specific business logic, so generic oracles such as "profit generation" or "reentrancy" miss them and automated tools produce high false-negative rates. The authors argue that different DeFi protocols often reuse the same underlying economic mechanisms — which they call **DeFi semantics**, for example proportional-share token accounting — and that the same attack scenarios recur across those mechanisms. Knowdit therefore summarizes recurring auditing knowledge from past human audit reports into a structured graph, then uses that knowledge to steer an LLM agent through specification writing, exploit synthesis, execution, and finding review on new projects.

## Key Contributions

1. **A knowledge-driven agentic workflow for smart contract auditing.** Knowdit maps a new project to high-level DeFi semantics and their historically linked vulnerability patterns, then iteratively converts each semantic–vulnerability pair into an executable proof-of-concept (PoC).
2. **An auditing knowledge graph built from 331 historical Code4rena contests.** The graph contains 650 deduplicated DeFi semantics across 13 categories (consolidated from 1,776 extracted candidates), 1,226 vulnerability patterns derived from 3,251 audit findings, and 1,954 validated links between semantics and patterns. Construction cost a one-time $1,336 in tokens, averaging $4.0 per contest.
3. **A bipartite schema separating DeFi Space from Vulnerability Space.** DeFi Space holds Solidity projects, business types, and DeFi semantics (linked by *belongs to*, *contains*, *underlies*); Vulnerability Space holds auditing findings, attack types, and vulnerability patterns (linked by *contributes to*, *exposes*, *involves*); the two are joined by *has* (project to finding) and *may introduce* (semantic to pattern).
4. **Strong empirical results with zero false positives on the held-out benchmark.** Knowdit detects all 21 high-severity and 57 of 63 medium-severity ground-truth vulnerabilities, fully covering 8 of 11 projects, and additionally finds 9 high- and 36 medium-severity previously unknown vulnerabilities in seven real-world projects.

## Main Findings

- **Dataset benchmark:** On AuditEval (11 Code4rena projects, 127 contracts, 57,810 lines of code, 21 high-severity and 63 medium-severity ground-truth vulnerabilities), Knowdit identifies all 21 high-severity vulnerabilities (100%) and 57 of 63 medium-severity vulnerabilities (90.5%) with no false positives.
- **Baseline comparison:** All baselines together detect between 1 and 24 vulnerabilities while producing considerably more false positives. Baselines evaluated are PropertyGPT, SmartInv, LLM-SmartAudit, GPTScan, and Codex-EVM (Codex version 0.136.0 with EVMBench prompts). PromFuzz was fully evaluated but excluded because it produced no findings on AuditEval.
- **Full coverage on 8 projects:** Knowdit recovers every ground-truth vulnerability with zero false positives on eight of the eleven evaluated projects.
- **Real-world discovery:** Applied to seven real-world projects, Knowdit finds 9 high-severity and 36 medium-severity previously unknown vulnerabilities, all confirmed and fixed by developers, including liquidity-draining issues, each accompanied by a concrete proof of exploitation.
- **Recurring root causes across protocols:** The motivating case pairs InsureDAO (2022) and Salty.IO (2024) — an insurance protocol and an AMM/liquidity protocol — which both implement proportional-share token accounting (shares follow s = aS/A, with S total shares and A total assets) and are both exposed to the first-depositor attack, where an adversary inflates the A/S ratio.
- **Knowledge graph quality:** Assessed by experienced auditors, the graph reaches high precision with substantial agreement (RQ2 ablation metrics and the quantitative agreement scores are not reported in the provided excerpt).
- **Model sensitivity:** The paper states Knowdit generalizes across different foundation models with a trade-off between detection effectiveness and cost efficiency (RQ4 numbers are not reported in the provided excerpt). The default configuration uses GPT-5.4.

## Methodology in Plain English

**Phase 1 — Building the knowledge graph.** The authors define a schema modeled on how a human auditor thinks: a project's *business type* (one of Lending, Dexes, Yield, Services, Derivatives, Yield Aggregator, Real World Assets, Stablecoins, Indexes, Insurance, NFT Marketplace, NFT Lending, Cross Chain) points to fine-grained *DeFi semantics* (e.g., token swapping), which in turn may introduce *vulnerability patterns*, which expose *attack types* (one of Access Control, Arithmetic, Block Manipulation, Cryptographic, Denial of Service, Reentrancy, Storage & Memory). Three LLM-driven stages populate it:

- *Stage I:* For each historical project, an LLM classifies its business type, extracts candidate DeFi semantics while stripping implementation details, then merges overlapping semantics into unified nodes. The example given merges UniswapV2 and UniswapV3 swapping into one "Constant Product" semantics node, but keeps Curve Tricrypto separate because it swaps three tokens rather than two.
- *Stage II:* For each human audit report, the LLM classifies attack types, abstracts each finding into a root-cause-level vulnerability pattern, and deduplicates patterns within the same attack type.
- *Stage III:* The LLM is prompted over the Cartesian product of semantic chunks and pattern chunks to emit *may introduce* links.

**Phase 2 — Auditing a new project.** Knowdit builds a **Repository Index** (call graph, state-variable reference graph, contract inheritance graph, and a source index by contract and function) and exposes it to agents as tools, which saves tokens. The **Knowledge Mapper** classifies the project's business types, extracts its DeFi semantics, matches them against the graph, and produces semantic–vulnerability pairs. Each pair then goes through a loop:

1. **Specification Generator** turns abstract knowledge into a project-specific specification with an Initial State, Pre-Vuln State, Post-Vuln State, and a transaction sequence, pointing at concrete functions and state variables.
2. **PoC Synthesizer** writes a Foundry test: Initial State becomes `setUp`, and the pre/post-vuln invariants become `require` statements acting as oracles.
3. **PoC Executor** runs the test and collects line coverage, state changes, and a full trace; PoCs that miss a minimum line coverage or never touch the specified state variables are dropped early to save tokens.
4. **Finding Reflector** compares the trace against the specification. Mismatches are classified as *Expected Behavior* (e.g., an `onlyOwner` revert) or *Problematic Specification or PoC*, and trigger regeneration. Matches are further checked against scope rules from the project README and Code4rena general rules (e.g., excluding fee-on-transfer token findings when the project disclaims such tokens), and overrides of those rules are supported.

**Reliability mechanisms.** Agents cannot emit final output directly; they must fill fields through dedicated `updateX` tools (e.g., `updatePreStateContract`) in an iterative drafting loop. Guardrails tied to the Repository Index reject fabricated contract or variable names that do not exist in the codebase.

**Experimental setup.** Knowdit is roughly 54.2k lines of Rust using Foundry as the execution engine. To avoid data leakage, contests before GPT-5.4's knowledge cutoff of 31/08/2025 build the graph, and contests from the following 6 months (unfiltered) form the evaluation set. All tools run in a shared Docker image on 16 CPU cores, two RTX A5000 GPUs, and 32GiB memory, each capped at $200 of LLM tokens per project, with every experiment repeated five times and unique findings aggregated. Two ablations isolate the components: **Knowdit w/o KG** removes the Knowledge Mapper, and **Knowdit w/ Codex** swaps Knowdit's agent framework for Codex-EVM while still retrieving from the same graph via the `bge-m3` embedding model.

## Why This Matters

**Impact on research.** The paper reframes smart contract vulnerability detection as a knowledge-transfer problem rather than a pattern-matching problem: instead of hand-writing templates, oracles, or invariants per project, it distills reusable abstractions from human audit reports and reuses them across protocols. The InsureDAO/Salty.IO pairing — the same root cause surfacing two years apart in unrelated business models — is direct evidence that such abstractions exist and are exploitable by an automated system. The zero-false-positive result on 11 held-out contests, against an industry-leading coding-agent harness, sets a demanding reference point for the field.

**Real-world applications:**

- Pre-deployment audits of DeFi protocols where business logic is custom and generic fuzzers miss logic bugs.
- Continuous monitoring or regression auditing of forked and code-reused contracts, which the paper cites as a major source of recurring exploits.
- Bug bounty and competitive audit triage, where an agent can produce reproducible Foundry PoCs alongside findings instead of prose-only reports.
- Insurance, underwriting, and risk scoring for DeFi protocols, using the semantic-level knowledge graph as a structured risk taxonomy.

**Industry relevance.** The authors cite CertiK's report that over $3.35 billion in cryptocurrency was stolen through hacks and scams in 2025. Knowdit's real-world run surfaced 9 high- and 36 medium-severity issues that developers confirmed and fixed, described as securing millions in liquidity — a direct commercial signal. The one-time $1,336 graph construction cost and $4.0 per contest average make the knowledge asset economically reusable across an audit firm's whole portfolio, and the $200-per-project token cap makes the per-audit cost comparable to other tooling.

## Future Directions

- **Self-growing knowledge.** The paper notes confirmed findings can be integrated back into the knowledge graph. How reliably that feedback loop avoids polluting the graph with project-specific noise — and how fast detection improves as a result — is unresolved.
- **Beyond Solidity and Code4rena.** The graph is built exclusively from Solidity projects and Code4rena reports, with business and attack types predefined from prior work. Extending it to non-EVM chains, Rust/Move contracts, or other audit platforms would test whether the "shared economic mechanism" insight generalizes.
- **Cost and model frontier.** The sensitivity study is framed as an effectiveness-versus-cost trade-off, and the entire benchmark uses GPT-5.4. Whether distilled or open-weight models can drive the same multi-agent loop at lower cost, and where the accuracy cliff sits, remains an open question.
- **Graph quality and coverage limits.** The 1,954 links are produced by LLM judgment over semantic–pattern chunk pairs, filtered by coverage-style heuristics. A systematic measurement of what the graph misses — and whether missed links explain the residual medium-severity failures (6 of 63) — is not established in the provided excerpt.

## Target Audience

Smart contract security auditors and audit-firm tooling engineers; LLM-agent researchers working on code intelligence and tool-use pipelines; DeFi protocol developers and risk teams evaluating pre-deployment review options; and academic researchers in program analysis and vulnerability detection who want a concrete, benchmarked example of combining knowledge graphs with agentic proof-of-concept synthesis. Readers should be comfortable with Solidity and EVM execution semantics to get full value from the methodology section.

## Authors’ abstract

Smart contracts govern billions of dollars in decentralized finance (DeFi), yet automated vulnerability detection remains challenging because many vulnerabilities are tightly coupled with project-specific business logic. We observe that recurring vulnerabilities across diverse DeFi business models often share the same underlying economic mechanisms, which we term DeFi semantics, and that capturing these shared abstractions can enable more systematic auditing. Building on…

[The full abstract on arXiv](https://arxiv.org/abs/2603.26270)

[Read the original paper](https://arxiv.org/abs/2603.26270)

---

Canonical: https://crypto.world/paper/knowdit-agentic-smart-contract-vulnerability-detection-with-auditing-2603.26270
