Research
Raven: Mining Defensive Patterns in Ethereum via Semantic Transaction Revert Invariants Categories
Security research on Ethereum mostly studies attacks that succeeded, while the defenses that actually stopped attacks — visible as reverted transactions — go largely unanalyzed.

- arXiv
- 2512.22616
- Published
- 2025-12-27
- Authors
- Mojtaba Eshghie, Melissa Mazura, Alexandre Bartel
AI summary
Overview
Research area: Blockchain security — smart contract invariants, transaction failure analysis, and unsupervised semantic clustering of Solidity/Vyper code (cs.CR).
Technical level: Intermediate. Readers need some familiarity with Ethereum transactions, Solidity guard statements (require / assert / if ... revert), and basic NLP clustering concepts, but the paper's core idea is accessible.
Scope: The paper presents Raven, a pipeline that links reverted Ethereum transactions to the source-code invariants that caused the revert, embeds those invariants with a fine-tuned BERT model, and clusters them into a catalog of defensive semantic categories — plus a case study using one discovered category as a fuzzing oracle against the Nomad Bridge incident.
What This Paper Is About
Security research on Ethereum mostly studies attacks that succeeded, while the defenses that actually stopped attacks — visible as reverted transactions — go largely unanalyzed. The authors argue that every transaction reverted by an invariant such as require(<invariant>) is a concrete instance of a working on-chain defense, and that these signals have never been systematically categorized. Raven's goal is to align reverted transactions to the exact invariant in the verified contract source, embed those invariants, and cluster them by semantic intent to build an empirical catalog of Ethereum's functioning defenses.
Key Contributions
- RavenBERT, a BERT-family embedding model (built by fine-tuning SmartBERT-v2) trained on revert-inducing invariants extracted from 100,000 failed Ethereum transactions, using contrastive learning over 16,470 automatically generated pairs.
- A methodology for configuring semantic clustering of invariants, including a grid search across four embeddings (TF-IDF, CodeBERT, SmartBERT-v2, RavenBERT), three clustering algorithms (K-Means, DBSCAN, HDBSCAN), two invariant "views" (predicate-only vs. predicate+message), and explicit admissibility criteria (granularity, coverage floor, stability) — 24 configurations in total.
- A manually labeled catalog of 19 invariant categories, six of which are reported as absent from prior invariant catalogs.
- A public dataset of 20,000 failed Ethereum transaction attributes, including the reversion invariant and its localization (file, function, contract), plus a case study showing a discovered category can serve as a working fuzzing oracle against the Nomad Bridge vulnerability.
Main Findings
- Best configuration quality: Raven's best admissible setting (RavenBERT embeddings, DBSCAN, predicate-only view) reached a Silhouette score of 0.93 and an S_Dbw score of 0.043, with 19 clusters at 51.86% coverage. The same model with error messages reached a Silhouette of 0.89, S_Dbw of 0.042, and 58.32% coverage.
- RavenBERT outperforms baselines: SmartBERT-v2 with DBSCAN and no messages reached a Silhouette of 0.86, 54.6% coverage, and S_Dbw of 0.069 — the next-best after RavenBERT. DBSCAN overall produced Silhouette scores of 0.79–0.93; HDBSCAN 0.59–0.85; K-Means 0.49–0.68 and was discarded because it over-divided into the maximum 99 clusters.
- Coverage–quality trade-off: Highest semantic quality occurs at roughly 52–58% coverage; other configurations push coverage to 72–86% while grouping semantically unrelated invariants. Adding the error message raised coverage from 52% to 58% but reduced cluster cohesion, which the authors attribute to stylistic/template-like variance (e.g., recurring OpenZeppelin strings like "caller is not the owner").
- Six previously unreported invariant categories: caller-provided slippage thresholds (relational, e.g.
amountOut >= amountOutMin), feature toggles, replay prevention, proof/signature verification, counters/nonces, and allow/ban/bot/whitelist gates. These are absent from Trace2Inv's 23 templates, InvCon+'s 14 templates, and the 15 formalized business logic properties modeled by DCR graphs. - Largest discovered cluster: caller-provided slippage thresholds, with 3,116 transactions (44.78% of clustered transactions in the labeled table). Across the 19 labeled clusters, the table reports 6,958 clustered transactions; the six new categories together account for 3,708 transaction reversions.
- Partial overlap validates the method: many clusters mapped onto known families (access control, budget/balance guards, time constraints, not-paused checks), supporting clustering validity while the new categories demonstrate that the design and grid search surface genuinely novel patterns.
- Case study success: the "proof/signature verification" category was converted into a fuzzing oracle in a Foundry-based fuzz test targeting the upgrade procedure, and it detected the Nomad Bridge incident's vulnerability.
- Data yield: from the 20,000-transaction evaluation sample, 12,222 invariants (61.1%) were extracted, 600 (3%) extraction failures occurred, and 7,178 (35.89%) reverts were non-invariant — dominated by "no source code found" (4,660, 23.3%) and "out of gas" (2,420, 12.1%). De-duplication reduced the sample to 727 unique invariant patterns (3.6%), of which 396 included error messages.
Methodology in Plain English
The authors started from Ethereum's failure population: 6,313,121 failed transactions between June 15, 2024 (block 20,100,158) and March 28, 2025 (block 22,145,000). For each transaction they pulled basic metadata and a preliminary failure reason, discarding out-of-gas and arithmetic failures (overflow, underflow, division by zero, which Solidity reverts automatically since 0.8.0). For the rest they used Tenderly's APIs to trace the exact stack frame that reverted, mapped the bytecode back to a source line via verified Etherscan source, and pulled out the guard statement — assert, require, or if (...) revert/throw — together with its optional error string.
Two disjoint datasets were built: 100,000 failed transactions yielding 1,932 unique invariants for fine-tuning, and a 20,000-transaction evaluation sample. The sample size was justified with Cochran's formula (95% confidence, ±1% margin, p = 0.5), which required a minimum of 9,604 transactions — the 20,000 used comfortably exceeds that.
For embeddings they compared a TF-IDF lexical baseline against three encoder-only BERT models. RavenBERT was produced by fine-tuning SmartBERT-v2 (pre-trained on 16,000 Solidity files) with contrastive learning: cosine similarity over the invariant set with a 0.8 threshold was used to auto-generate 16,470 positive and negative pairs. Masked Language Modeling fine-tuning was tried first but performed slightly worse than the base model, so it was abandoned. Vectors were L2-normalized so cosine distance behaves as a proper metric.
Clustering used K-Means, DBSCAN, and HDBSCAN with a grid search over the ranges in the paper (k from 8 to 100; DBSCAN ε from 0.1 to 5 and min_samples from 10 to 15; HDBSCAN ε from 0.1 to 1 and min_cluster_size from 10 to 15). A configuration was only "admissible" if it produced 8–100 clusters, covered at least 50% of invariants with non-noise labels, and gave stable metrics across two seeded re-runs. Ties were broken by minimizing S_Dbw, then maximizing Silhouette, then coverage, then preferring fewer clusters. Because no ground-truth taxonomy exists, external metrics like ARI were unavailable, so four intrinsic metrics were used jointly: Silhouette, S_Dbw, coverage, and admissibility.
Finally, two authors (with a third as tiebreaker) manually reviewed every cluster, labeling it meaningful if a unifying logic applied to at least 90% of its invariants, mapping labels against three prior catalogs to identify novel categories, and checking that clusters did not overlap. PCA plots of the two leading components were used as a geometric sanity check on the metric results.
Why This Matters
Impact on research: The paper reframes reverted transactions from noise into a positive security signal — a source of empirically grounded oracles rather than post-mortem evidence of failure. It provides the first study on clustering transaction-revert invariants and shows that existing invariant catalogs (Trace2Inv, InvCon+, DCR-graph properties) are incomplete with respect to what contracts actually enforce in production.
Real-world applications:
- Data-driven fuzzing oracles: mined invariant categories can be turned into test assertions, as demonstrated against the Nomad Bridge upgrade vulnerability.
- Runtime monitoring and anomaly detection: categories like feature toggles, replay prevention, and counters/nonces give operators concrete predicates to monitor.
- Audit and code review checklists: the 19-category catalog can be used as a template set that reflects what real contracts defend against, including patterns auditors may not otherwise look for.
- Empirical benchmark construction: the released dataset of 20,000 failed transactions with invariant localization supports reproducible evaluation of invariant miners and verifiers.
Industry relevance: Smart contract platforms, audit firms, and DeFi protocol teams can use the catalog to compare their own defenses against ecosystem practice, and the tooling (public GitHub repository, RavenBERT and dataset on Hugging Face) is directly reusable.
Future Directions
- Scaling beyond the sample: the evaluation used 20,000 transactions extracted with a sampling strategy; extending dynamic analysis to the full 6,313,121-transaction population (and beyond) is an open feasibility question the authors frame as resource-bound.
- Improving coverage without sacrificing cohesion: the 52–58% coverage of the preferred configuration means a large share of invariants remain noise; whether better representations or semi-supervised labeling can raise coverage while retaining cluster purity is unresolved.
- Closing the unverified-source gap: "no source code found" was the single largest non-invariant failure category (4,660, 23.3%), so defenses in unverified contracts are invisible to this pipeline.
- Turning categories into tools: only one newly discovered category (proof/signature verification) was taken through to a fuzzing oracle; generalizing the other five — feature toggles, replay prevention, counters/nonces, list gating, and slippage thresholds — into automated detectors is the obvious next step.
Target Audience
Smart contract security researchers and tooling engineers, invariant-mining and formal-verification groups, and applied NLP researchers interested in domain-specific embedding fine-tuning. Auditors and DeFi protocol engineers will find the 19-category catalog and the case study practically useful, while graduate students will find the grid-search-and-admissibility methodology a reusable template for unsupervised code clustering without ground truth.
Authors’ abstract
We frame Ethereum transactions reverted by invariants-require(<invariant>)/ assert(<invariant>)/if (<invariant>) revert statements in the contract implementation-as a positive signal of active on-chain defenses. Despite their value, the defensive patterns in these transactions remain undiscovered and underutilized in security research. We present Raven, a framework that aligns reverted transactions to the invariant causing the reversion in the smart…
The full abstract on arXiv