EVM datasets: columns from 9db54d3 (logs_bloom, access_list, access_list_size) missing below a per-dataset boundary on base-mainnet / arbitrum-one / avalanche-mainnet

[dzhelezov]

Summary

The transactions columns added in 9db54d3 ("add new columns to evm block and transaction", 2026-04-29) — logs_bloom, access_list, access_list_size (and the blob fields) — are missing below a per-dataset block boundary on at least base-mainnet, arbitrum-one, and avalanche-mainnet. A query that requests them over a historical range fails with HTTP 400:

couldn't parse request: column 'logs_bloom' is not found in 'transactions'
couldn't parse request: column 'access_list_size' is not found in 'transactions'

(the accessList probe surfaces access_list_size first because it is the weight column — crates/query/src/query/eth.rs).

The same query succeeds on ethereum-mainnet, polygon-mainnet, and binance-mainnet across their full ranges, and — importantly — succeeds on the affected datasets too for recent ranges: the datasets are mixed-schema, split at the date each writer picked up the new schema.

Reproduction (public portal, no key)

# 400 below the boundary:
curl -s -X POST "https://portal.sqd.dev/datasets/base-mainnet/finalized-stream" \
  -H 'content-type: application/json' \
  -d '{"type":"evm","fromBlock":20000000,"toBlock":20000000,"fields":{"transaction":{"logsBloom":true}},"transactions":[{}]}'

# 200 above it (same dataset, same query):
curl -s -X POST "https://portal.sqd.dev/datasets/base-mainnet/finalized-stream" \
  -H 'content-type: application/json' \
  -d '{"type":"evm","fromBlock":48300000,"toBlock":48300000,"fields":{"transaction":{"logsBloom":true}},"transactions":[{}]}'

Bisected availability boundaries (2026-07-08):

dataset	400 below	200 from	boundary block (bisected)	≈ boundary date
base-mainnet	20.0M	48.3M	~45,398,144	2026-05-01
arbitrum-one	300M	481M	~457–458.4M	2026-05-01
avalanche-mainnet	50M	89.7M	~89.08–89.39M	2026-06-29 … 07-03

The chains themselves carry the data (verified against RPC: accessList on typed transactions, logsBloom on receipts), so this is purely a dataset-schema matter.

Root cause

1. 9db54d3 added the columns to the transactions table (crates/data/src/evm/tables/transaction.rs); they are written unconditionally by the builder.
2. Parquet chunk schemas are frozen at write time and no schema version is stamped into the chunk layout — the chunk's parquet footer is the schema.
3. The scan layer validates every requested column against each chunk's physical footer and raises ColumnDoesNotExist on a miss (crates/query/src/scan/parquet/file.rs → crates/query/src/scan/errors.rs), which the worker classifies as BadRequest and the portal relays as HTTP 400.
4. Datasets whose chunks were fully re-generated after the schema change serve the columns everywhere; datasets still on their original chunk sets serve them only from the date their writer picked up the new schema (2026-04-30 rollout; avalanche later, matching its stack migration in early July). Hence the mixed-schema boundaries above.

Note the graceful-null machinery from ad62dc2 (NET-92: handle missing parquet columns as nulls) exists, but set_nullable currently has no callers for the EVM tables, so the strict per-chunk validation is what users see.

Secondary failure mode (worker version skew)

On the affected datasets, requests for the new fields over new (post-boundary) ranges intermittently fail with a different 400 — unknown field 'accessList' — when served by workers older than v2.11.0 (the first worker release carrying the 9db54d3 schema). Observed nondeterministically (2/5 identical requests on arbitrum-one). Datasets re-generated post-change appear to be pinned to a minimum worker version; the affected three are not, so old workers still serve them.

Impact

• Consumers that need receipt-level fields per transaction (e.g. Ponder-style indexers with includeTransactionReceipts: true, which need transaction.logsBloom for byte-complete receipt rows) cannot backfill historical ranges on three major chains. Downstream report: Portal dataset gap: transactions.logs_bloom missing on base-mainnet / arbitrum-one / avalanche-mainnet — includeTransactionReceipts backfills fail fast (by design) subsquid-labs/portal-ponder#83.
• accessList consumers silently degrade (nullable field), see docs(validation): L-base-logs finding — base-mainnet dataset lacks transactions.access_list (#83-family, benign) subsquid-labs/portal-ponder#90.
• Because availability is range-dependent and /datasets/{ds}/metadata exposes no column information, clients can only discover this by probing for 400s at their specific fromBlock.

Suggested fix, in order of leverage

1. Immediate (config-only): pin the minimum worker version (≥ v2.11.0) for base-mainnet, arbitrum-one, avalanche-mainnet — eliminates the nondeterministic unknown field failures on post-boundary ranges today, and makes the remaining behavior deterministic (clean per-range 400s).
2. The real fix: schedule these three datasets into the ongoing chunk re-generation rotation (as already done for ethereum/polygon/binance post-schema-change). Since the raw data is retained, this is a raw→parquet reconversion — no RPC re-ingest — and the precedents suggest weeks-scale wall-clock per chain. base-mainnet is the highest-value target (largest downstream indexer demand for receipts).
3. Policy option (cheaper, trade-offs): register the 9db54d3 columns as nullable via the existing NET-92 machinery so pre-boundary chunks return nulls instead of 400. This is a deliberate policy reversal (nulls can mask genuinely-present chain data — presumably why the defaults were removed), so if adopted it would be better as an explicit opt-in query flag (e.g. "nullableMissing": true) than as a default.
4. DX: expose per-dataset column availability — ideally with the range boundary — via /datasets/{ds}/metadata, so clients can plan field selection instead of discovering gaps through 400 probes at arbitrary blocks.

Happy to provide the full probe logs / bisection data.