Skip to content
Crypto.world

Research

Early Rug Pull Detection on Solana

Tests whether Solana transfer graphs actually beat non-graph models in a token's first minutes, where rug warnings must be issued with almost no history.

Early Rug Pull Detection on Solana
arXiv
2611.13342
Published
2026-11-09
Authors
Saba Heydaridoost, Kasra Kashani, Mohammad Mowlavi Sorond, Behnam Bahrak

AI summary

Overview

Research area: Blockchain security and decentralized finance (DeFi) fraud detection, specifically early detection of "rug pull" scams on the Solana blockchain using graph neural networks versus gradient-boosted tree models.

Technical level: Advanced. The paper assumes familiarity with graph neural networks (GCN, GraphSAGE, GAT), gradient boosting (XGBoost), average precision, bootstrap confidence intervals, and multiple-testing correction.

Scope (one sentence): A controlled, leakage-aware benchmark that tests whether the token-transfer graph lets a GNN flag Solana rug pulls earlier than a non-graph model given identical features and identical tokens, across three pre-registered observation windows of 12.5, 35.5, and 93 minutes.

What This Paper Is About

On Solana, anyone can create a tradable token in a few steps for a fraction of a cent (about $0.00025 per transaction on average), and rug pulls often unfold within minutes of creation, so any useful warning must be built from a very short trading history. Prior work on EVM chains suggests that graph neural networks over transaction graphs beat feature-based detectors, but those gains were usually measured against baselines with different features, different token populations, or different prediction points, so it is unclear whether the graph structure itself is responsible. The authors ask a practical question for platforms that list new tokens: does the transfer graph let a rug pull be detected earlier than a model without it?

Key Contributions

  1. A leakage-aware Solana benchmark and evaluation protocol for early rug-pull detection, combining cutoff-separated operational labels, observation windows fixed before data collection, a chronological train/validation/test split with a label-availability embargo, and two evaluation scopes (full population and a fixed panel of tokens valid in all windows).

  2. Evidence that rug-pull and non-rug transfer graphs already differ structurally within the first 12.5 minutes, most strongly in receiver-side centralization, reciprocity, and graph size/density.

  3. A pre-specified comparison of graph and non-graph models with identical fourteen features and identical test tokens across three observation windows, with ablations and a set of model variants (direction-aware, bagged, stacked, global-graph, heterogeneous-graph).

  4. Practical guidance for early-warning mechanisms in Web3 interfaces for newly launched Solana tokens, including the argument that graph-based claims need paired comparisons, multiple seeds, correction for multiple testing, and adequately sized test sets.

Main Findings

  • Early structural signal is real. In the training partition of the original cohort (579, 632, and 599 graphs at q1, q2, q3), rug-pull and non-rug graphs differ significantly after Benjamini–Hochberg correction in every window. Rug-pull graphs involve more endpoints (median 145 vs. 58.5 at q1) but are sparser, with more one-directional transfers; non-rug activity concentrates around a few dominant receivers. The largest differences are in maximum in-degree centrality and maximum PageRank (rank-biserial r = −0.42 for both at q1), and non-rug graphs are also more reciprocal with higher maximum betweenness.

  • No supported earlier-detection advantage (RQ1, primary analysis). On the panel test set (n = 136, 38 positives, prior-probability classifier AP = 0.279), none of the nine pre-specified contrasts — each GNN at q1 versus XGBoost at q1, q2, or q3 — has a Bonferroni-adjusted 95% interval that excludes zero. Test AP values were: GCN .477/.473/.460; GraphSAGE .522/.528/.536; GAT .457/.513/.481; XGBoost .464/.452/.393 across q1/q2/q3. The larger differences at later windows reflect only XGBoost's declining AP on the same tokens, since GNN predictions are fixed at q1.

  • Longer observation does not help XGBoost on the fixed panel. XGBoost's test AP falls from 0.464 at q1 to 0.393 at q3 on the same 704-token panel (rug-pull rate 34.8%). The panel is selective: requiring validity at q3 removes 386 q1 tokens with a rug-pull rate of 74.9%.

  • Graph models often rank better at the top, but not significantly (RQ2). Using validation-selected models and seed-ensemble predictions, the selected GNN achieves higher test AP in 5 of 6 scope-by-window cells at similar ROC-AUC; the exception is panel q3, where FT-Transformer reaches .542 versus GCN's .489. The largest gain is full q2 (GraphSAGE .597/.726 vs. FT-Transformer .489/.689, ΔAP +.108 [.019, .184]). Across 10 comparisons including added XGBoost cells, three have unadjusted intervals excluding zero, but none survives Holm correction (smallest adjusted p = 0.14).

  • The tests are underpowered, so null results are inconclusive. A power analysis based on bootstrap standard errors indicates the test sets can only detect AP differences of about 0.10–0.18 at 80% power with α = 0.05, and correction raises these thresholds further.

  • Validation scores badly overstate deployed performance. AP drops from 0.82–0.91 on validation to 0.45–0.63 on test, and ROC-AUC from 0.84–0.91 to 0.65–0.73. Re-weighting validation to test prevalence explains only 16–41% of the AP gap and none of the ROC-AUC drop. The token population shifts markedly: pump.fun mints make up 12–14% of training, 36–40% of validation, and 63–67% of test samples, and transfer k-NN AUC falls from 0.77–0.79 on validation to 0.53–0.67 on test. A stratified random split instead of the chronological split would have inflated test ROC-AUC by 0.07–0.17.

  • Graph-summary features help non-graph models modestly. Adding the nine graph-summary features to the five token-level features raises median test AP of non-graph models by 0.029 in both scopes, but only 10 of 135 configurations in the full scope and 6 of 135 in the panel have unadjusted intervals excluding zero. Graph-summary features alone perform about as well as token-level features in the full scope (median ΔAP = +0.004). In panel q1, a token-only random forest reaches the highest ROC-AUC of all models (0.744) but a lower AP than the selected GNN.

  • Graph-only GNNs are competitive. GNNs using only the transfer graph achieve panel test AP of 0.507, 0.539, and 0.450 at q1–q3, versus 0.522, 0.513, and 0.460 for the full models, so the additional features provide no consistent improvement in these runs.

  • Variants do not change the picture. Direction-aware GraphSAGE/GCN give ΔAP between −0.030 and +0.033; a single global graph gives ensemble ΔAP between −0.060 and +0.022; bagging GraphSAGE improves ensemble AP in 4 of 6 cells by up to 0.069 but no interval excludes zero; stacking the graph score into gradient boosting or XGBoost lowers AP in 11 of 12 comparisons; an adaptation of HNN4RP to Solana without its contract-code embedding is worse than the reference model in every cell. Adding missingness indicators changes median test AP by less than 0.01.

  • Data availability correlates with the outcome. Complete cases have higher rug-pull rates than the original cohort, and graph-bearing samples make up 46.4%, 53.1%, and 56.4% of the cohort at q1–q3, with rug-pull rates of 43.6%, 39.2%, and 31.9%.

Methodology in Plain English

The authors start from 2,700 mint addresses randomly sampled with a fixed seed from the 2024 portion of SolRPDS (a public Solana rug-pull dataset), after discarding empty entries and keeping one entry per mint from 27,752 candidates. They then collect their own data: one-minute liquidity and USD price histories from Birdeye for labeling and token-level features, and raw token transfers from Solana RPC.

Defining a rug pull. A token is labeled positive only if three conditions hold within a nominal 24-hour horizon: liquidity falls more than 80% below its highest earlier level at some time τ; the lowest price after τ is more than 90% below the highest price before τ; and liquidity stays at or below the 20% remaining level for at least 80% of the 30 minutes after τ and is still below it at the end. All thresholds were fixed before labeling, and varying the price threshold between 0.80 and 0.95 or the persistence threshold between 0.60 and 0.90 changes the number of positives by at most 3.6%.

Three decision points. Predictions are made at cutoffs of 750 s, 2,127 s, and 5,585 s after creation (about 12.5, 35.5, and 93 minutes) — the 25th, 50th, and 75th percentiles of the rug-pull lifespan distribution reported by Chen et al., fixed before data collection. Only features observable at or before the cutoff are used. Samples whose liquidity drop is already visible inside the window are excluded (128, 321, and 590 tokens respectively), as are drops within 60 s of the cutoff, because they cannot be placed reliably at one-minute resolution. This leaves 1,961, 1,798, and 1,573 samples with rug-pull rates of 39.0%, 35.9%, and 30.1%.

Features. Each sample gets five token-level features (buy- and sell-volume change, unique-wallet count and change, liquidity at cutoff) and nine graph-summary features (transfer count, endpoints, distinct ordered pairs, senders, receivers, density, average degree, earliest and mean relative transfer time), for fourteen in total. Each token's transfers form a directed temporal multigraph with six per-endpoint features; transfer amounts are deliberately excluded because raw token amounts are not comparable across tokens.

Models. Non-graph baselines include a prior-probability classifier, logistic regression, decision tree, random forest, gradient boosting, XGBoost, an MLP, FT-Transformer, and AutoInt, each trained on four feature views. The GNNs (GCN, GraphSAGE, GAT) use message passing over each token's graph with pooling to a graph embedding, then concatenate that embedding with the same fourteen features, so both families see identical inputs and the GNN only adds endpoint-level structure.

Evaluation discipline. Tokens are split chronologically (oldest 70% training, next 15% validation, newest 15% test) so that no model learns from tokens newer than the test set, and tokens whose labels only become available after the next partition begins are embargoed (9, 8, and 7 tokens). Hyperparameters are tuned with Optuna TPE searches (60 trials for classical models, 40 for neural and graph models) maximizing validation ROC-AUC, and each configuration is retrained with five seeds. The primary analysis — nine comparisons of GNNs at q1 against XGBoost at q1, q2, q3 — was fixed before the test partition was touched, uses 10,000 paired bootstrap samples with within-class resampling, and applies a Bonferroni correction for a joint 95% confidence level; all other comparisons are exploratory, with Holm-adjusted p-values for within-window contrasts.

Why This Matters

Impact on research. The paper's central methodological point is that reported graph-model gains in fraud detection are often unattributable, because graph and non-graph baselines differ in features, token populations, or prediction points. By holding features and test tokens constant and pre-registering the comparison, the authors provide a template for how graph-based security claims should be evaluated. The finding that validation scores of 0.82–0.91 AP collapse to 0.45–0.63 under a chronological split is a concrete warning against random splits in this literature.

Real-world applications:

  • Launchpads and DEX interfaces could use minute-scale model scores to prioritize newly created tokens for review, with the caveat that scores should not be presented as reliable probabilities or definitive judgments.
  • Wallet software could surface warnings for tokens whose first 12.5 minutes of transfer activity resemble the rug-pull pattern (many endpoints, sparse one-directional flows, low receiver concentration).
  • Token-analytics sites could monitor input coverage alongside model scores, since complete cases and graph-bearing tokens have systematically higher rug-pull rates and missingness changes over time.
  • Risk and compliance teams at exchanges could adopt the paper's label definition and embargo scheme as a reproducible way to backtest listing-time risk models.

Industry relevance. Solana's low fees and shared token program (no per-token contract code) make contract-analysis approaches from Ethereum inapplicable, so platforms must rely on behavioral signals that exist only after trading begins. The paper's practical message is that early signals are real and models can rank tokens usefully from the first minutes, but the incremental value of graph-based scoring remains unresolved — and claims to the contrary need paired comparisons, multiple seeds, and larger test sets.

Future Directions

  1. Larger evaluation sets. The authors explicitly call for bigger cohorts: current test sets contain 136–159 tokens and can only detect AP differences of roughly 0.10–0.18, so whether a small but genuine graph benefit exists is still open.

  2. Richer graph representations. Temporal GNNs with explicit edge-time message passing, and graphs that incorporate transfer amounts (excluded here because raw amounts are not comparable across tokens), are natural extensions of the mostly topological graphs used in this study.

  3. Broader label coverage. The operational definition only captures rug pulls that drain liquidity within 24 hours; schemes without a liquidity drain, such as freeze-authority abuse (Chen et al. found nine such cases among 100 tokens not flagged within a comparable 24-hour window), are out of scope.

  4. Cross-token structure and distribution shift. A single global graph over all tokens did not help at q1, and the strong temporal shift in token population (pump.fun share rising from 12–14% in training to 63–67% in test) raises the question of how models should be retrained, recalibrated, and monitored as the launch ecosystem evolves.

Target Audience

Researchers and graduate students working on blockchain security, financial fraud detection, and graph machine learning, particularly those interested in rigorous evaluation of GNNs against strong tabular baselines. It is also directly useful to engineers and data scientists building risk systems at launchpads, DEX interfaces, wallets, and token-analytics platforms on Solana, and to reviewers or practitioners who need a model for how to design leakage-aware, temporally valid benchmarks for adversarial financial data.

Authors’ abstract

Web3 platforms that help users discover and trade newly launched crypto tokens, including launchpads, decentralized-exchange (DEX) interfaces, wallets, and token-analytics sites, must assess tokens before their legitimacy is clear. On Solana, where token creation is inexpensive and tokens use standard programs rather than custom per-token smart contracts, rug pulls can unfold within minutes. Effective warnings must therefore rely on limited histories. Although graph neural…

The full abstract on arXiv
Read the original paper