# When Verified Source Becomes Attack Input: Defending Smart Contracts Against LLM-Based Vulnerability Scanning

Crypto.world

> DeFi projects publish and verify their Solidity source code on blockchain explorers like Etherscan to build trust and enable audits. The paper argues that LLM agents turn this transparency mechanism…

**arXiv**: 2608.28400

**Published**: 2026-08-28

**Authors**: Mingyuan Huang, Zimo Ji, Yifan Mo, Shuai Wang

## AI summary

## Overview

**Research area:** Blockchain security — specifically smart contract deployment mechanics and the security of verified source-code disclosure against automated LLM-based vulnerability scanning (arXiv category: Cryptography and Security).

**Technical level:** Intermediate. The paper assumes familiarity with Ethereum-style contracts, proxies, `delegatecall`, and `CREATE2`, but explains each mechanism it uses.

**Scope:** The paper proposes and evaluates DeLLMGuard, a deployment-level framework that keeps smart contract source code publicly verified while making it substantially harder for LLM agents to locate the root cause of real vulnerabilities, tested on 387 real world vulnerable contracts with three frontier LLM agents.

## What This Paper Is About

DeFi projects publish and verify their Solidity source code on blockchain explorers like Etherscan to build trust and enable audits. The paper argues that LLM agents turn this transparency mechanism into a scalable attack input: an agent can pull verified source from a public entry address and scan for exploitable bugs automatically and cheaply. The goal is a defense that preserves public source disclosure and authorized auditing, yet disrupts the direct link between the address a user (or attacker) starts from and the source code that reveals the business logic.

## Key Contributions

1. **Problem formulation.** The authors define the security problem of defending publicly disclosed smart contracts against malicious LLM-based vulnerability scanning while preserving source disclosure in real-world blockchain environments, and they lay out four requirements (R1 real-world applicability, R2 proxy identification, R3 verifiable business preservation, R4 source code availability).
2. **DeLLMGuard, a deployment-level defense framework.** The paper claims this is the first deployment-level framework countering automated LLM-based vulnerability scanning while keeping public source disclosure and security auditing intact. It uses multi-address deployment so the disclosed source address is separated from the contracts involved in runtime execution.
3. **Four composable guard components plus a Verification Layer.** Factory Indirection Component (FIC), Delegate Layering Component (DLC), Proxy Diversification Component (PDC), and Comment Perturbation Component (CPC), checked by a Verification Layer that validates deployment relations, routing, runtime bytecode, source tokens, and state changes. A separate Bytecode Inline Component (BIC) serves as a closed-source comparison baseline.
4. **Empirical evaluation on 387 real-world vulnerable contracts.** Three LLM agents (GLM 5.2, GPT 5.5, Claude Opus 4.8) are run in an environment derived from Anthropic's SCONE-bench, with overall root-cause correctness dropping from 23.5% to 6.6%, and DeLLMGuard outperforming the closed-source BIC baseline on the primary non-proxy set.

## Main Findings

- **Large reduction in root-cause detection.** DeLLMGuard reduces overall root-cause correctness from **23.5% to 6.6%** across the evaluated agents, and the paper reports reducing root-cause detection by **up to 87%**.
- **It beats hiding the source.** On the primary non-proxy dataset, DeLLMGuard matches or outperforms the closed-source Bytecode Inline baseline (BIC), even though BIC withholds the original high-level business source entirely and therefore fails requirement R4.
- **Failure is about context reconstruction, not raw capability.** Trace and ablation analyses show that agents often do recover downstream contracts but still fail to identify the vulnerability. The authors conclude that cross-contract recovery remains a major challenge for automated LLM scanning.
- **Effectiveness survives a fully informed attacker.** In the adversarial robustness question (RQ4), the defense remains substantial even when the agent is informed of the complete DeLLMGuard pipeline and transformation method. The threat model explicitly does not assume the mechanism is secret.
- **Bytecode decompilation does not neutralize the defense.** Decompilation agents can recover local instructions such as `SLOAD`, `DELEGATECALL`, and `CREATE2`, but can still misinterpret their semantics or the cross-contract relations they encode; correct analysis requires linking bytecode with storage and transaction information.
- **Benchmark construction and retention.** Starting from 417 SCONE-bench tasks, 398 have publicly disclosed source usable by all candidate transformations. After sandbox setup, 387 cases were successfully set up; only 11 (3%) could not be evaluated because of failed constructors or missing data. The frozen set is 387 cases: **341 non-proxy and 46 proxy**, covering incidents from 2020 through 2026 across Ethereum mainnet, BNB Smart Chain, Base, and Arbitrum.
- **Judge reliability was measured.** Using Cochran's formula, 194 agent reports were sampled at a 95% confidence level and manually judged; agreement with the `deepseek-v4-flash` LLM judge gave a Cohen's κ of **0.767**.
- **Not reported in the available content:** per-model breakdowns, per-component ablation numbers, gas or deployment-cost overhead, and the exact per-run time budget (the text is truncated at "45-min"). These should be treated as unreported rather than assumed.

## Methodology in Plain English

The researchers start from a catalogue of documented DeFi hacks (DeFiHackLabs incidents that were incorporated into SCONE-bench), then reconstruct each victim contract's address, verified source, compiler metadata, and bytecode from explorers and RPC endpoints.

For each case they build two parallel worlds at the same historical block: an **untreated control** and one or more **transformed deployments**. Because transformed contracts do not exist on the real chain, they built a sandbox called **PARSEC** (Paired Address-based Replay Sandbox) that forks the historical chain state with Anvil, records code, storage, balance, nonce and transaction data into a replay bundle with a SHA-256 manifest, and then serves the agent a local RPC with no upstream access. Agents start with only a public entry address and must discover everything else.

The transformation itself is deliberately mundane — it uses only standard EVM features, no language or VM changes:

- **Proxy Analyzer** decides whether the entry address is a proxy, recovers the implementation address, and records compiler versions, so each guard is applied in the right place.
- **DLC** puts a small guard contract at the entry address that loads a target from a dedicated routing slot and forwards calls via `delegatecall`, so the business bytecode runs in the guard's storage context while the source remains disclosed elsewhere.
- **PDC** writes misleading "shadow" identities into unused, standardized proxy-related slots (EIP-1967 admin, EIP-1967 beacon, EIP-1822 PROXIABLE, legacy ZeppelinOS admin), creating competing candidate implementations without touching the real implementation slot or forwarding logic.
- **FIC** deploys the business contract through a factory using `CREATE` or `CREATE2`, so the disclosed source sits at the factory while the entry address only carries business bytecode.
- **CPC** changes non-executable comments only — a lightweight text-level nudge.
- **BIC** (baseline only) returns the original runtime bytecode directly from a wrapper constructor, pre-empting explorer verification so the original Solidity is never disclosed.

Before any transformed contract reaches the LLM, the **Verification Layer** compares an expected deployment graph from the transformation manifest against an evidence graph reconstructed from bytecode, storage, transaction traces, and compiler outputs. A deployment is accepted only if deployment, routing, runtime, source, and state predicates all pass — including byte-identical business runtime bytecode and a matching source token sequence (comments and formatting ignored).

Evaluation scores agents with an LLM judge given one hidden root-cause record and one agent report, labeling results Correct, Partially correct, Incorrect, or Not detected. The primary metric is Correct; Correct + Partial Correct is also reported. Crucially, the paper measures **vulnerability discovery only**, not end-to-end exploitation, and conservatively treats complete root-cause identification as attacker success while deliberately not building an auto-exploitation pipeline into the sandbox.

## Why This Matters

**Impact on research.** The paper reframes source-code transparency as an attack surface and shows that a defense can be built at the deployment layer rather than in the model or the analysis tooling. That is a different lever from most LLM-security work, which assumes the defender controls the model or the execution environment — assumptions the authors argue do not hold on a public blockchain. It also produces a reproducible paired-control benchmark methodology (PARSEC) and an explicit, expert-validated judging protocol with a reported κ.

**Real-world applications:**

- **DeFi protocol launches:** teams can deploy with FIC, DLC, PDC, or a composite configuration while still verifying source on explorers, raising the cost of automated mass scanning.
- **Security auditing workflows:** authorized auditors receive the mapping and full source package directly, so legitimate review is unaffected while permissionless automated analysis becomes more expensive.
- **Explorer and verification tooling:** the paper exposes how standardized proxy slots (EIP-1967, EIP-1822, ZeppelinOS) act as machine-readable hints for automated analysis, which matters for anyone building detection heuristics.
- **Post-deployment monitoring and incident response:** teams can reason about how much on-chain evidence remains publicly reconstructible versus what must be shared privately.

**Industry relevance.** The authors cite Halborn's report of indications that an LLM assistant was used in the 2025 Balancer exploit, causing over $120 million in losses, and note that Code4rena explicitly discourages LLM-generated audit submissions because of their high invalid-finding ratio. Immutability plus pseudonymous ownership means AI-assisted findings cannot be routed into the kind of patch-and-release process that traditional software uses — which is exactly the gap a deployment-level defense targets.

## Future Directions

- **Cross-contract recovery as the open problem.** The trace and ablation results show agents frequently reach downstream contracts but still fail at the vulnerability, so improving an agent's ability to reconstruct multi-address execution context is the obvious next step — and the obvious next attack.
- **Stronger adversary configurations.** The study uses three models with one run per test case and a fixed harness; testing repeated attempts, larger budgets, and agents explicitly trained or prompted on the DeLLMGuard pipeline would probe how durable the reported reduction is.
- **Composite configurations and cost accounting.** The components are composable, but the available content does not report gas overhead, deployment cost, or per-component ablation figures, which are needed before practitioners can weigh protection against operational expense.
- **Generalization and scope limits.** The benchmark is 387 EVM cases (341 non-proxy, 46 proxy) spanning 2020–2026, and 11 cases (3%) failed to reproduce in simulation; extending to more chains, non-EVM runtimes, and contracts where source is only partially disclosed remains open.

## Target Audience

Smart contract developers and protocol maintainers who publish verified source, blockchain security researchers working on LLM-agent threat models, auditors who need to understand how disclosure mechanics interact with automation, and tooling engineers building proxy or verification infrastructure. Readers need moderate blockchain background; the paper explains its primitives but assumes comfort with proxies, `delegatecall`, and `CREATE2`.

## Authors’ abstract

Smart contracts are financial programs deployed on blockchains to manage digital assets. To build trust with users and investors, smart contract projects typically publish their source code on blockchain explorers and verify it against the deployed bytecode, making the on-chain program accessible through a human-readable implementation. However, LLM agents are changing the threat model of this disclosure mechanism. By leveraging publicly disclosed source code, recent agent…

[The full abstract on arXiv](https://arxiv.org/abs/2608.28400)

[Read the original paper](https://arxiv.org/abs/2608.28400)

---

Canonical: https://crypto.world/paper/when-verified-source-becomes-attack-input-defending-smart-contracts-2608.28400
