Skip to content

test: cover mixed-format reorder column reads#366

Merged
JingsongLi merged 1 commit into
apache:mainfrom
liujiwen-up:fix/258-mixed-format-reorder-move-column
Jun 16, 2026
Merged

test: cover mixed-format reorder column reads#366
JingsongLi merged 1 commit into
apache:mainfrom
liujiwen-up:fix/258-mixed-format-reorder-move-column

Conversation

@liujiwen-up

@liujiwen-up liujiwen-up commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: #258

Add focused Spark-provisioned integration coverage for mixed-format schema evolution reads after ALTER COLUMN ... FIRST/AFTER.

This PR intentionally covers only the reorder/move-column case to avoid overlapping with #354, which covers mixed-format add-column and type-promotion reads.

Brief change log

  • Add a Spark fixture with Parquet, ORC, and Avro files written across column reorder/move schema versions.
  • Add an integration test verifying full reads expose the current table schema order.
  • Verify mixed-format reads map values by field id rather than physical column position.
  • Verify projected reads preserve caller-specified projection order after column reorder/move.

Tests

  • cargo fmt: passed
  • git diff --check: passed
  • cargo test -p paimon-integration-tests --test read_tables --no-run: passed

API and Format

No API or storage format changes. This PR only adds Spark provisioning and Rust integration test coverage.

Documentation

No documentation update required.

@liujiwen-up liujiwen-up changed the title test: cover mixed-format schema evolution reads test: add mixed-format schema evolution reads Jun 5, 2026
@liujiwen-up liujiwen-up closed this Jun 5, 2026
@liujiwen-up liujiwen-up reopened this Jun 7, 2026
@liujiwen-up liujiwen-up force-pushed the fix/258-mixed-format-reorder-move-column branch from 7c64a66 to ca012be Compare June 7, 2026 15:36
@liujiwen-up liujiwen-up changed the title test: add mixed-format schema evolution reads test: cover mixed-format reorder column reads Jun 7, 2026
@liujiwen-up liujiwen-up force-pushed the fix/258-mixed-format-reorder-move-column branch from ca012be to f2cc921 Compare June 9, 2026 06:30
@JingsongLi

Copy link
Copy Markdown
Contributor

Please rebase main.

@JingsongLi JingsongLi force-pushed the fix/258-mixed-format-reorder-move-column branch from f2cc921 to 536394a Compare June 16, 2026 04:09
&plan,
&["avro", "orc", "parquet"],
"mixed_format_schema_evolution_reorder_move_column",
);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: can we also assert multiple schema ids here, like the adjacent mixed-format schema evolution tests? This would make the test prove it is exercising the historical-schema path, not only mixed file formats.

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi merged commit 8c0aeaf into apache:main Jun 16, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants