# Ira: Efficient Transaction Replay for Distributed Systems

Crypto.world

> In primary-backup replication, the primary executes a batch of transactions and backups must re-execute it to verify and stay consistent. The backups do this blindly: they only learn which keys a…

**arXiv**: 2601.21286

**Published**: 2026-01-29

**Authors**: Adithya Bhat, Harshal Bhadreshkumar Shah, Mohsen Minaei

## AI summary

Ira: Efficient Transaction Replay for Distributed Systems

## Overview

**Research area:** Distributed systems — specifically primary-backup (state machine) replication, with Ethereum block validation as the case study.

**Technical level:** Intermediate. Readers should be comfortable with the idea of replicated state machines, caches, and key-value storage; the paper supplies its own Ethereum background section.

**Scope:** The paper proposes Ira, a general framework in which a primary node ships compact "hints" describing the keys a transaction batch touched, so backups can prefetch exactly the state they need; it then builds and evaluates Ira-L, a concrete instantiation for Ethereum block replay.

## What This Paper Is About

In primary-backup replication, the primary executes a batch of transactions and backups must re-execute it to verify and stay consistent. The backups do this blindly: they only learn which keys a transaction needs at the moment it needs them, causing disk reads and poor cache decisions. The paper's observation is that the primary already knows the full access pattern and can cheaply pass that knowledge forward. The goal is to make backup replay dramatically faster without changing correctness guarantees.

## Key Contributions

1. **Measurement study of Ethereum execution.** The authors instrumented reth (a production Rust Ethereum client) and re-executed 100,800 consecutive mainnet blocks (blocks 24,019,447–24,120,246, spanning December 15–29, 2025), recording 1,192,888,639 state operations across 14 distinct operation types. They identify I/O as the dominant cost and caching as the main lever.
2. **Hint-based replay architecture.** A general design in which the primary records the access set during execution and transmits it alongside the transaction batch, letting backups prefetch the entire working set before replay. Hints are *advisory*: a missing, corrupt, or incomplete hint degrades performance but never correctness, which permits incremental deployment.
3. **Ira-L, an Ethereum-specific instantiation.** Per-block hints containing storage keys tagged with a one-byte *source annotation* (main state table, zero, or historical changeset), plus account addresses and bytecode addresses. Backups sort all keys lexicographically and walk the B-tree with a forward cursor, converting random I/O into sequential scan and achieving zero cold misses.
4. **Implementation and evaluation.** Ira is implemented by modifying reth and evaluated on the full two-week dataset, reporting hint-generation overhead, compressed hint size, and replay speedups at several prefetch thread counts.

## Main Findings

- **I/O dominates block execution:** I/O accounts for 67.9% of total execution time versus 32.1% for EVM compute (median per-block I/O share 66.8%, 5th–95th percentile range 55.6%–80.2%), an I/O-to-compute ratio of 2.1:1.
- **Storage dominates I/O:** SLOAD/SSTORE make up 75.6% of all state accesses (901,775,607 operations), calls 23.6% (281,933,575), account metadata 0.6% (6,751,623), and creation/destruction 0.2% (2,427,834). The storage read-write ratio is 3.4:1 (696,683,837 SLOADs vs. 205,091,770 SSTOREs).
- **Strong intra-block locality:** The global reuse factor is 25.74x (901,775,607 operations over 35,035,731 unique keys). Within a block, 65.5% of keys are accessed more than once, and 77.7% of storage accesses hit keys already touched earlier in the same block (only 200,839,090 of 901,876,518 accesses are first accesses). The most frequently accessed key appears 23,406 times in a single block.
- **Extreme access concentration:** The top 3 contracts (USDT, USDC, Compound) account for 42.2% of storage operations and the top 10 for 63.6%; among 273,223 contracts, the top 10,000 account for 97.5%. The top 1.23% of keys (430,873 keys with 100+ accesses) account for 84.0% of storage operations, while 64% of keys are accessed only once or twice and contribute 3.6% of accesses.
- **Keys are largely ephemeral:** 72.4% of keys appear in only one block, 21.4% in 2–5 blocks, and only 1.0% persist across more than 50 blocks. Consecutive-block overlap averages 12.4% (median 11.7%, maximum 100.0%), which is why LRU-style policies cannot predict the next block's needs.
- **Working sets are bounded and hintable:** The median block performs 8,629 operations, of which 6,587 are storage operations touching 1,806 unique keys; the 95th percentile is 3,920 unique keys. 58% of blocks access fewer than 2,000 unique keys and 84% fewer than 3,000, with only 1.2% exceeding 5,000. Hinting 2,000 keys costs at most 101 KB uncompressed at 52 bytes per key.
- **Source annotations pay off:** 46.6% of storage keys resolve to PlainState, 10.5% to Zero, and 43.0% to Changeset, so 57.0% of keys bypass the history index entirely (per-block median 55.3%). PlainState keys need one B-tree seek instead of two, and Zero keys need no I/O at all.
- **Replay speedup:** Hint-driven prefetching speeds up aggregate replay by 5.2x with a single prefetch thread and by 23.6x with 16 threads (abstract); the contributions section and Section 4 cite 24.9x faster replay with 10.9% hint-generation overhead.
- **Hint cost on the primary:** Hint construction (940 s, 6.0% of aggregate execution time) plus serialization (778 s, 5.0%) totals 1,719 s, or 10.9%. End-to-end wall time rises from 23,250 s for the unmodified baseline to 29,889 s, a 28.6% increase. Per-block, the median hint fraction is 12.1% and the 95th percentile 22.1%.
- **Hint size is small:** Hints add a median of 47 KB compressed metadata per block, roughly 5% of block payload; Section 4.1 reports a median of 46.5 KB at a 2.17x compression ratio against a ~2 MB block payload. Each storage key serializes to 53 bytes (20-byte address + 32-byte slot + 1-byte annotation) and each address key to 20 bytes.
- **Construction scales with block size, serialization does not:** Construction has a median of 14.4 ms and 95th percentile of 33.8 ms, scaling at roughly 8.1 microseconds per key (r = 0.79), while serialization clusters near 12.5 ms regardless of block size (r = 0.03), dominated by the fixed cost of flushing a database write transaction.

## Methodology in Plain English

The authors first measured before they designed. They hooked into reth's EVM inspector interface to watch every state-touching instruction during real block execution, using stack introspection to pull out the target address and storage slot at the moment of each access. Replaying archival state with a write-through cache kept intra-block semantics correct, and blocks were processed in parallel and stored as Parquet with Zstandard compression.

Those traces produced six findings about Ethereum's access patterns, which in turn dictated the design of Ira-L. On the primary side, the same inspector collects three sets during normal execution — storage keys, account addresses, and bytecode addresses — and tags each storage key with a one-byte source annotation determined by consulting the history index. The result is compressed and written to a key-value database indexed by block number.

On the backup side, the key trick is sorting: all keys from a batch (default 32 blocks) are sorted lexicographically by (address, slot) so the cursor moves monotonically through the B-tree and touches each page at most once. This replaces random seeks that would otherwise traverse O(log N) pages each. A warm-up phase prefetches roughly 3,000 blocks into an 8 GB in-memory buffer, then a dedicated prefetcher thread feeds the executor through a bounded channel so wall-clock time is bounded by max(prefetch, execution) rather than their sum. Each block's cache is thrown away afterward, so memory stays bounded to one block's working set and no LRU eviction is needed.

Correctness is preserved because hints are advisory. EVM execution is deterministic, so the backup would access the same keys anyway; Zstandard's built-in integrity check catches corrupt hints, and any coverage gap triggers graceful fallback to reth's standard execution path. Benchmarks ran on an Apple M1 Pro (10 cores: 8 performance + 2 efficiency) with 32 GB unified memory and an 8 TB NVMe SSD, always starting from a purged, cold OS page cache.

## Why This Matters

**Impact on research.** The paper reframes replay inefficiency as an *information asymmetry* problem rather than a storage-engineering problem: the primary knows the future access pattern and discards it. That framing generalizes beyond Ethereum to any primary-backup system, and the advisory-hint design means the idea can be layered onto existing protocols without touching their safety arguments. The measurement study is also independently useful as a characterization of real Ethereum mainnet workloads.

**Real-world applications:**

- **Blockchain validation:** Faster block replay directly narrows the window in which validators must verify a block before attesting, and the authors argue this weakens the verifier's dilemma and improves denial-of-service resilience.
- **Replicated databases and Raft-based stores:** The same hint mechanism applies to any primary that executes before replicas re-execute (the paper discusses adaptation in an appendix).
- **Streaming replication and distributed key-value stores:** Read-heavy replicated workloads with skewed, rapidly changing hot sets are exactly the regime where the measured 12.4% cross-block overlap defeats static caching.
- **Read replicas and audit/verification nodes:** Any node that re-executes a workload to check someone else's results rather than to produce them can benefit.

**Industry relevance.** The evaluation is against reth, a production Rust client, on two weeks of real mainnet blocks, and the overhead is measured in wall time rather than simulated cost. A 10.9% cost on the primary to buy roughly 5.2x to 23.6x on backups is a favorable trade in any replication topology where there are more backups than primaries. The hint's small footprint — a median of 47 KB, about 5% of block payload — makes the bandwidth cost easy to justify.

## Future Directions

- **Hiding hint cost entirely:** The authors note that construction and serialization are independent of execution correctness and could be overlapped with the next block's processing; the reported median 12.1% per-block hint fraction reflects a conservative, fully sequential measurement, so the practical overhead under pipelining is an open question.
- **Adapting beyond Ethereum:** The paper defers to an appendix the question of how Ira maps onto Raft-based databases, streaming replication, and distributed key-value stores, where the notion of a "source annotation" has no direct analogue and the hint schema would need rethinking.
- **Adversarial and incentive questions:** The threat model explicitly allows a Byzantine primary that omits hints, sends failing-integrity hints, or sends hints with missing or spurious keys. The paper leaves open how to identify and penalize peers that repeatedly deliver invalid hints, and how to incentivize correct hinting.
- **Confidential settings:** Hints reveal storage-access patterns. This is harmless on Ethereum, where access patterns are public, but the authors flag it as unsuitable for permissioned settings with confidential transactions — a design constraint that would need addressing before the technique crosses into enterprise deployments.

## Target Audience

This paper suits distributed-systems researchers and engineers working on replication, consensus latency, or state machine replication; blockchain client developers interested in execution-layer performance and caching; and performance engineers who deal with read-heavy key-value workloads where access patterns are unpredictable across time windows. Graduate students looking for a clean example of "measure first, then design" — where six empirical findings directly motivate a specific algorithmic choice — will also find it instructive. Readers unfamiliar with Ethereum will need the primer in Section 2.2, but no deep protocol expertise is assumed.

## Authors’ abstract

In primary-backup replication, the consensus latency is bounded by the time for backup nodes to replay (re-execute) transactions proposed by the primary. Our key insight is that the primary, having already executed transactions, possesses knowledge of future access patterns which is the information needed for optimal replay by the backups. In this work, we present Ira, a framework to accelerate backup replay by transmitting compact hints alongside transaction batches. We use…

[The full abstract on arXiv](https://arxiv.org/abs/2601.21286)

[Read the original paper](https://arxiv.org/abs/2601.21286)

---

Canonical: https://crypto.world/paper/ira-efficient-transaction-replay-for-distributed-systems-2601.21286
