Research
Solidity Meets LLMs: A Transformer-Based Approach to Smart Contract Vulnerability Detection
Smart contracts on Ethereum are immutable once deployed, so a bug cannot be patched after the fact — the 2016 DAO reentrancy attack, which drained 3.6 million ETH (approximately 70 million USD at the…

- arXiv
- 2609.27091
- Published
- 2026-09-22
- Authors
- Djamel Eddine Hakim Ghorab, Farid Mokhati, Mostafa Anouar Ghorab
AI summary
Overview
Research area: Blockchain security, specifically smart contract vulnerability detection using natural language processing (transformer/LLM) techniques on Solidity source code.
Technical level: Intermediate. Readers should know roughly what a smart contract is and what fine-tuning a pre-trained language model involves; no deep formal-methods background is required.
One-sentence scope: The paper fine-tunes a BERT-based model to classify snippets of Solidity source code as vulnerable or safe, using labels generated automatically by two existing analysis tools as ground truth.
What This Paper Is About
Smart contracts on Ethereum are immutable once deployed, so a bug cannot be patched after the fact — the 2016 DAO reentrancy attack, which drained 3.6 million ETH (approximately 70 million USD at the time), is the paper's motivating example. Most existing detectors need fully compiled bytecode, a complete abstract syntax tree (AST), or formally specified rules, which makes them awkward to use on partial or in-progress code inside a development environment. The authors ask whether a pre-trained transformer language model can instead learn Solidity vulnerability patterns directly from source-code fragments and flag them as vulnerable or safe.
Key Contributions
- A taxonomy of Solidity vulnerabilities organized into five categories — cross-call (e.g., reentrancy), denial of service, arithmetic, access control, and bad randomness — with the impact of each described.
- An automated ground-truth labeling pipeline that combines Slither and SolScan, applying the rule that a contract is "safe" only if both tools agree it is safe, while a detection by either tool is treated as a vulnerability. Output is structured as
contract_name<SEP>Safety<SEP>Type<SEP>Lines, and vulnerable line numbers are extracted and cleaned (comments, extra spaces, and new lines removed). - A fine-tuned BERT binary classifier built with the Hugging Face
BertTokenizer, trained on a 70% / 15% / 15% train-validation-test split to classify Solidity code fragments as vulnerable or safe. - A comparison of existing Solidity analysis tools (SmartPulse, ContractFuzzer, SolScan, Osiris, SmartCheck, Slither) on detection level, bytecode/AST requirements, static vs. dynamic type, and whether the tool can handle unfinished code — positioning the proposed approach as usable on partial code.
Main Findings
- Reported headline performance: The abstract states the fine-tuned model achieves an F1 score of 92%.
- Per-epoch results (five epochs): F1-score rises from 0.923976 in epoch 1 to 0.943279 in epoch 5; accuracy and recall both reach 0.94375 by epoch 4 and stay there in epoch 5; precision peaks at 0.951237 in epoch 4 and is 0.949393 in epoch 5.
- Loss decreases steadily: Training loss falls from 0.551955 (epoch 1) to 0.323758 (epoch 5); validation loss falls from 0.382489 to 0.224617, which the authors read as evidence of learning without severe overfitting.
- Other metrics improve in tandem: MCC reaches 0.887873 and Roc_AUC 0.940091 by epoch 5, versus 0.847273 and 0.924189 in epoch 1.
- Only static comparison tools were evaluated: Among the tools compared, SolScan is the only one marked as able to handle unfinished code; SmartCheck, Slither, SmartPulse, ContractFuzzer, and Osiris are all marked as unable to.
- Dataset size is not reported: The paper describes sources (contracts from previous works and Etherscan-verified contracts) and the split percentages, but gives no count of contracts or code fragments.
Methodology in Plain English
The authors first gathered raw Solidity contract files from prior research datasets and from contracts verified on Etherscan. Because manual labeling is slow, they let two existing scanners do the labeling. One scanner, Slither, analyzes the contract with all of its modules, then a second pass runs specific modules to identify the vulnerability type according to the authors' taxonomy and records the line numbers flagged as the source. The other, a modified version of SolScan (a pattern-recognition tool), saves detected line numbers and vulnerability types directly. The two results are merged: if either tool flags the contract, the flagged lines are cut out and labeled vulnerable; if both call it safe, snippets from that contract are labeled safe. Extracted snippets are stripped of comments, extra whitespace, and line breaks.
The resulting text snippets are split 70% for training, 15% for validation, and 15% for testing, tokenized with the BERT tokenizer, and used to fine-tune a BERT model for binary classification (vulnerable vs. safe) over five epochs. Training ran on an A100-SXM4-40GB GPU with 50GB of RAM.
Why This Matters
Impact on research: The work tests whether general-purpose language-model pretraining transfers to a specialized, security-critical programming language, and it proposes a cheap way to build labeled data without expensive expert annotation — a bottleneck that limits most machine-learning approaches to smart contract security.
Real-world applications:
- IDE or editor plugins that flag risky Solidity fragments while code is still being written, since the approach does not require compilation or a complete AST.
- Pre-audit triage for development teams, prioritizing which contracts or functions deserve human auditor attention.
- Continuous-integration checks on pull requests in smart contract repositories.
- Educational tooling that points new Solidity developers at risky patterns such as unchecked external calls before deployment.
Industry relevance: Because deployed contracts are immutable and often hold significant financial value, catching flaws before deployment has direct economic consequences. An approach that works on partial code lowers the barrier to early-stage security checking compared with tools that require bytecode, an AST, or a full contract.
Future Directions
- Higher-quality datasets: The authors propose adding corrected vulnerable examples, misleading code, and edge cases to reduce false positives.
- Better ground-truth labels: Combining static and dynamic analysis tools with expert manual review, since the current labels are inherited from Slither and SolScan rather than verified by humans.
- Fine-grained, multi-class detection: Moving beyond binary vulnerable/safe output to classify the specific vulnerability type using a full taxonomy, so developers learn what the flaw actually is.
- Automatic remediation: Using more capable LLMs not only to detect flaws but to generate secure fixes, connecting detection to repair.
Target Audience
Researchers and graduate students working at the intersection of machine learning and software security; smart contract developers and auditors curious about lightweight, source-level detection that works on incomplete code; and security tooling engineers evaluating whether transformer models can complement or replace rule-based scanners in a development workflow. Readers looking for a rigorous benchmark against existing detectors or a large-scale empirical study will find this paper preliminary — it reports a single model configuration and does not report dataset size or compare accuracy head-to-head against Slither or SolScan on the same test set.
Authors’ abstract
The growing adoption of blockchain technologies, particularly the Ethereum platform, has amplified the critical role of smart contracts in decentralized applications. However, the increasing complexity and financial value of these contracts make them prime targets for cyber attacks. In this work, we present a transformer-based approach for the detection of vulnerabilities in smart contract fragments written in Solidity. Leveraging the representational power of pre-trained…
The full abstract on arXiv