fix(evm): bound store-cache depth amplification from stacked EVM snapshots#3713
Conversation
…shots Each successful EVM call frame stacks a new nested CacheMultiStore layer (DBImpl.Snapshot), and geth never signals success so the layers are never discarded. After N sequential successful calls the current context sits atop ~N nested cachekv layers, and Cosmos reads at that depth (e.g. the distribution `rewards()` precompile -> IterateDelegations + per-delegation reward math) walk every layer. That makes a linear number of reads O(N^2) in wall-clock while gas stays linear. Fix, entirely opt-in so non-EVM code is unchanged: - cachekv / giga store: add frozen+dirty state, Freeze()/Unfreeze(), and a memoized readThroughParent() that skips runs of frozen empty layers on the Get path. iterator() also returns the parent iterator directly when the layer is empty in-range (closing the unused cache iterator), collapsing the cacheMergeIterator chain. Reads become O(1) in depth instead of O(depth). - cachemulti: propagate Freeze()/Unfreeze() to sub-stores via a shared flag so lazily-materialized stores inherit the state. - DBImpl.Snapshot (x/evm/state and the giga fork) freezes the just-superseded layer, never the base layer (snapshottedCtxs[0], the flush target read by GetCommittedState). RevertToSnapshot unfreezes the re-exposed layer so a writable top is never treated as a skippable frozen layer. Safety: a layer only takes writes while it is the newest layer; skips are gated on `frozen && !dirty`, so a re-exposed-and-written layer is always consulted again. Benchmark: deep-stack Get goes from ~16us (depth 1024) to a flat ~134ns. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3713 +/- ##
==========================================
- Coverage 59.30% 58.38% -0.93%
==========================================
Files 2274 2187 -87
Lines 188706 178980 -9726
==========================================
- Hits 111917 104500 -7417
+ Misses 66716 65206 -1510
+ Partials 10073 9274 -799
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
PR SummaryMedium Risk Overview
New tests cover equivalence, shadowing, revert→write→snapshot, stale memo after unfreeze, and deep-stack benchmarks. Reviewed by Cursor Bugbot for commit b2d9282. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
A carefully-scoped, opt-in optimization that freezes superseded EVM snapshot cache layers so deep reads skip empty layers, fixing O(N²) read amplification; the core logic is correct and well-tested for the single-DBImpl case, with a few non-blocking concerns around the cross-Copy stale-memo edge and the breadth of the iterator fast path. No blockers found.
Findings: 0 blocking | 6 non-blocking | 2 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- Cross-
Copy()deep-ancestor staleness:readThroughParent's memo is only bypassed when the immediate parent's livefrozen/dirtybit changes. In thestatedb.Copy()path (e.g. block_trace_profiled.go), layers are shared between DBImpls. If one DBImpl reverts to and writes into a deep shared ancestor A while another (copy) still sees its immediate parent as frozen-empty, the copy's memoized skip past A would return stale data (miss A's new writes). The PR's own comment/description acknowledges this and argues safety ('the EVM never reverts below a tx's starting layer; trace copies are read-only'), but none of the new tests exercise the two-DBImpl shared-chain deep-revert case — only the immediate-parent case (TestUnfreezeBypassesStaleSkipMemo). Recommend adding a test that reverts/writes a non-immediate ancestor shared via Copy, or documenting/asserting the 'no revert below tx start' invariant more concretely. - Asymmetry: the giga store (
giga/deps/store/cachekv.go) gains the frozen-skip only on the Get path (readThroughParent) but itsiterator()was not given the fast-path/skip treatment thatsei-cosmos/store/cachekv/store.goreceived, so deep giga iteration remains O(depth). This is correctness-safe (giga backs evm/bank and is point-read heavy) but worth noting so the asymmetry is intentional, not an oversight. - Second-opinion passes produced no findings:
codex-review.mdstates 'No material issues found in the PR diff.' andcursor-review.mdis empty. Nothing to merge from either; noting for completeness. - Minor:
cachemulti.Store.Freeze()/Unfreeze()iteratecms.storesundermu.RLock()but iteratecms.gigaStoreswith no lock. This appears safe today (gigaStores is populated at construction and only mutated by SetGigaKVStores, not concurrently with Freeze), but the inconsistency is worth a comment given the file's emphasis on the OCC scheduler calling these concurrently. - 2 suggestion(s)/nit(s) flagged inline on specific lines.
| // Cheap gate: only nested cache layers can be skipped. The common single-layer | ||
| // case (parent is an iavl/dbadapter store) never enters the skip path. | ||
| cp, ok := store.parent.(*Store) | ||
| if !ok || !cp.frozen.Load() || cp.dirty.Load() { |
There was a problem hiding this comment.
[suggestion] This gate only checks the immediate parent's live frozen/dirty bit. Once a memo is stored (line 106), it can skip multiple deeper frozen-empty ancestors; if one of those non-immediate ancestors is later unfrozen and written (possible when layers are shared across DBImpls via statedb.Copy()), a store whose immediate parent is still frozen-empty will keep using the stale memo and miss those writes. Within a single DBImpl this can't happen (revert discards everything above the target), but the shared-Copy case relies on the unproven 'EVM never reverts below a tx's starting layer' invariant. Consider adding a test that exercises a deep revert+write on a Copy-shared chain, or invalidating the memo more defensively.
| // layers pass through, nested empty layers collapse the whole chain to the base | ||
| // iterator. Note dirtyItems materializes in-range deletes into sortedCache, so a | ||
| // non-Valid cache correctly means "no sets and no deletes in range". | ||
| if !cache.Valid() { |
There was a problem hiding this comment.
[suggestion] This !cache.Valid() fast path returns the raw parent iterator (instead of wrapping in a CacheMergeIterator) for every cachekv layer whose in-range cache is empty — not just frozen EVM layers. Unlike the Get path (gated on frozen), this is a global behavioral change: callers now receive the parent store's iterator type directly, so anything relying on the merge wrapper's Domain() or on the iterator identity/type for an empty-in-range layer will see different behavior. It looks behavior-preserving for iteration order/keys/values, but the change is broader than the PR's 'entirely opt-in' framing suggests — worth calling out explicitly.
There was a problem hiding this comment.
I didn't find any bugs, but this changes core cachekv/cachemulti store read/iterator paths plus the EVM Snapshot/RevertToSnapshot lifecycle in both x/evm/state and the giga fork, so it warrants a human look before merging.
What was reviewed
- The
Freeze/Unfreeze/readThroughParentskip logic in bothcachekvcopies and the iterator fast-path (parent-return when the cache is empty in-range). cachemultifrozen-flag propagation to lazily-materialized substores (shared*atomic.BoolinStorevalue copies).- The
Snapshot(freeze superseded layer, skip base at index 0) andRevertToSnapshot(unfreeze re-exposed top) wiring in bothx/evm/state/state.goandgiga/deps/xevm/state/state.go. - The concurrent-
Copy()trace path: sharedsnapshottedCtxslayers between original and traced copy, and the description's claim that the immediate-parent unfreeze gate keeps memoized skip pointers correct.
Worth a human look because the invariant ("a frozen layer never gains writes as long as it stays superseded") is load-bearing across Snapshot/Revert/Copy and any future call site that stacks or re-exposes cache layers.
Extended reasoning...
Overview
This PR adds an opt-in Freeze/Unfreeze mechanism to cachekv (both sei-cosmos/store/cachekv and giga/deps/store) and propagates it through cachemulti. It uses that mechanism in DBImpl.Snapshot/RevertToSnapshot (both the main and giga forks) to short-circuit reads through empty superseded snapshot layers so that a Cosmos read at EVM call depth N doesn't walk N layers. The read path adds a memoized readThroughParent() that skips runs of frozen+empty layers; the iterator path returns the parent iterator directly when the cache has no in-range writes/deletes. Test coverage is broad (equivalence, shadowing, revert→write→snapshot, stale-memo-after-unfreeze, Write() reset, deep-stack benchmark).
Security risks
Not an auth/crypto/permissions surface. The main risk is correctness: a wrongly-taken skip would silently drop writes from reads and change state root computation, which would be app-hash-breaking. The PR is labeled non-app-hash-breaking, consistent with the design (skips are only taken when frozen && !dirty, and dirty is set on any Set/Delete). The concurrent-Copy() interaction is the subtlest case — the design gates on the immediate parent's live frozen bit, but shared ancestors between original and copy still need the "EVM never reverts below a tx's starting layer" invariant to hold for the whole memoized skip chain, which I could not fully verify from the diff alone.
Level of scrutiny
High. cachekv and cachemulti are core Cosmos SDK store primitives — every read and iteration in the app funnels through them, and every module (not just EVM) is affected any time freeze flags get propagated. Even though the code path is opt-in, mistakes in the concurrency/memoization logic would corrupt read semantics under load, and would likely not surface in unit tests. The Snapshot/Revert wiring in DBImpl is on the EVM tx execution hot path.
Other factors
- Complexity: seven files, ~500 lines of net change touching atomics, memoization, iterator plumbing, and shared-flag propagation.
- The PR description is thorough and the invariants are documented in-code, which helps a human review land, but the reasoning about the
Copy()trace path and re-exposed shared layers is the kind of thing that benefits from a second pair of eyes. - Bugbot (external tool) also flagged this as "Medium Risk"; that's not a blocker but is consistent with my read.
- CI passes; codecov reports 92% patch coverage; buf is green.
* main: fix(evm): bound store-cache depth amplification from stacked EVM snapshots (#3713) Fix address distro bug (#3716) fix(giga): route EVM validation failures to v2 fallback (CON-368) (#3753) feat(seidb): composite + replay digest modes and migration-boundary omission for evm-logical-digest (#3711) feat(scripts): add SC read-path RPC probe (consistency check + load) (#3751) fix(feegrant): bound DenomsSubsetOf cost and cap allowance denoms (#3710)
Problem
Each successful EVM call frame stacks a new nested
CacheMultiStorelayer inDBImpl.Snapshot(). The sei go-ethereum fork only callsRevertToSnapshoton failure — the success-pathDiscardSnapshotis a commented-out TODO, andvm.StateDBexposes no discard hook — so geth cannot tell sei-chain a frame succeeded and the layers are never released.After
Nsequential successful calls the current context sits atop ~Nnestedcachekvlayers. A Cosmos read at that depth then walks every layer:rewards()precompile (reachable viaCALL/STATICCALL) →DelegationTotalRewards→stakingKeeper.IterateDelegations(an iterator through the merge-iterator chain) plus ~8–10 per-delegation point reads (Validator,IncrementValidatorPeriod,GetDelegatorStartingInfo,GetValidatorHistoricalRewards).Nlayers. Done at growing depth across the attacker's call loop, a linear number of reads becomes O(N²) wall-clock while gas stays linear.Staking/distribution route through the regular nested
cachekv(not the giga store), and those layers are empty for EVM txs — yet the merge iterators andgetFromCachestill traverse them.Fix
Entirely opt-in, so all non-EVM code paths are unchanged (they never call
Freeze()):cachekv/ giga store — addfrozen+dirtystate,Freeze()/Unfreeze(), and a memoizedreadThroughParent()that skips runs of frozen empty layers on theGetpath.iterator()returns the parent iterator directly when the layer is empty in-range (closing the unused cache iterator), collapsing thecacheMergeIteratorchain. Reads become O(1) in depth.cachemulti— propagateFreeze()/Unfreeze()to sub-stores via a shared flag so lazily-materialized stores inherit the state.DBImpl.Snapshot(x/evm/stateand the giga fork) freezes the just-superseded layer — never the base layer (snapshottedCtxs[0], the flush target read directly byGetCommittedState).RevertToSnapshotunfreezes the re-exposed layer, since a writable top must never be treated as a skippable frozen layer.Why it's correct
A layer only receives writes while it is the newest layer; once superseded it is frozen and, if empty, stays empty. Skips are gated on
frozen && !dirty, so a re-exposed-and-written layer (revert → write → snapshot) is always consulted again. Within one DBImpl,RevertToSnapshotalso discards every layer above the target (so no live skip memo can go stale) and now unfreezes the target. The concurrentstatedb.Copy()trace path shares layers, but is safe because the EVM never reverts below a tx's starting layer, and the unfreeze gate covers the immediate-parent case.Verification
frozen_skip_test.goincachekvand giga store): read/iterate equivalence through deep empty stacks, shadowing/deletes through frozen layers (must not skip), the exact revert → write → snapshot sequence, unfreeze bypassing a stale skip memo, andWrite()reset.Getgoes from O(depth) (251 ns → 3.9 µs → 16.2 µs at depth 16/256/1024) to a flat ~134 ns — ~120× faster at depth 1024.-race:cachekv,cachemulti, both giga stores,x/evm/state,giga/deps/xevm/state,x/evm/keeper, the distribution + staking precompiles, and the full giga integration suite.gofmt,goimports,go vetclean.🤖 Generated with Claude Code