The Ledger
Study finds LLMs often misdiagnose Solana bridge flaws
A study of five open-weight language models found they correctly identified 24% of vulnerable Solana bridge source-code tests without comments. Restoring original comments raised the diagnosis rate to 60%, while transaction-based results varied by attack scenario.

Diagnosis lagged behind alarms
Large language models correctly diagnosed just 24% of vulnerable Solana bridge source-code tests when comments were removed, according to a preprint posted on arXiv. Restoring the programs’ original comments raised source-code diagnosis to 60%, the paper reports, although some comments described the missing checks. The researchers tested whether models could identify planted flaws from source code and recognize attacks in transaction records.
The preprint was submitted to arXiv on November 16, 2026, by Yasir Rahman, Ishraq Samiha, Shahnewaz Karim Sakib and Anindya Bijoy Das. The authors built a native Rust lock-and-mint bridge on one local Solana validator and planted four exploitable variants. Two modeled mechanisms associated with the Cashio and Wormhole incidents; the other two represented vulnerability classes documented in Solana security audits.
Four planted attack patterns
The test cases covered mint confusion, spoofed instructions-sysvar data, a missing signer check for an administrator change, and an arbitrary cross-program invocation. Each exploit succeeded against its vulnerable variant and was rejected by a secured baseline. The scenarios tested whether a model could identify the particular missing validation, rather than merely label a program or session as suspicious.
The study evaluated six configurations of five open-weight models, collecting 16,800 responses across source-code and transaction-record tests. Two LLM graders assessed 7,200 answers on vulnerable inputs; they agreed on 7,119, or 98.88%, and the researchers resolved the remaining disagreements against predefined criteria. The paper counted an alarm and a correct diagnosis separately: a model had to name the planted flaw, and for source code, the affected function.
High alarm rates, uneven results
Without comments, five of the six configurations flagged 97% to 100% of vulnerable source-code inputs. But the paper reports that those configurations also produced 373 to 400 false positives across 400 secure-bridge evaluations. Correctly warning that something might be wrong therefore did not necessarily mean the model had identified the vulnerability the researchers had planted.
Transaction-record results depended on the attack. Five configurations correctly diagnosed the missing-signer exploit in 96 to 100 of 100 runs, but legitimate signed administrator changes also attracted alarms. For the other three scenarios—mint confusion, sysvar spoofing and arbitrary cross-program invocation—diagnosis averaged 6%, according to the study.
Comments changed the source-code results substantially: correct diagnoses increased from 574 of 2,400 runs without comments to 1,433 of 2,400 with original comments. The authors note that comments can reveal missing validation, so the higher score does not establish that models inferred the flaw from executable logic. The test focused on program-level validation in a local setup; it did not test agreement between independent blockchains, remote finality or the security of a distributed relayer network.
Sources
- Original source · Preprint LLMs as Validators for Cross-Chain Bridges: Evaluating Source-Level and Transaction-Level Attack Detection on Solana arXiv:2611.21777
Related stories
- Researchers measure a hash-based verifier for Bitcoin bridges
- Solana study finds no clear early advantage for graph-based rug detection
- Space and Time introduces private-data platform for AI
- Scroll Details Permissioned AI Network Plan and Ethereum Migration
- Solana Validators Face Linux Flaw Rated 9.8 as Ubuntu Patch Waits