Skip to content

feat: add Mosaic data file reader#386

Merged
JingsongLi merged 1 commit into
apache:mainfrom
QuakeWang:mosaic-read
Jun 16, 2026
Merged

feat: add Mosaic data file reader#386
JingsongLi merged 1 commit into
apache:mainfrom
QuakeWang:mosaic-read

Conversation

@QuakeWang

Copy link
Copy Markdown
Member

Purpose

Linked issue: #378

Paimon Rust currently does not recognize .mosaic data files, so Rust clients cannot read tables written with Mosaic by Java Paimon, PyPaimon, or paimon-mosaic-core.

This PR adds the first read-only Mosaic reader foundation behind the mosaic feature.

Brief change log

  • Add optional paimon-mosaic-core dependency behind the mosaic feature.
  • Add .mosaic reader dispatch when the feature is enabled.
  • Implement a read-only Mosaic format reader using a memory-backed InputFile.
  • Preserve projection order, empty projection, row selection, and unsupported-type errors.
  • Add DataFusion feature passthrough and include mosaic in CI feature checks.

Tests

  • cargo fmt --all -- --check
  • cargo test -p paimon --features mosaic arrow::format::mosaic
  • cargo build --features fulltext,vortex,mosaic
  • cargo clippy --all-targets --workspace --features fulltext,vortex,mosaic -- -D warnings
  • cargo test -p paimon --all-targets --features fulltext,vortex,mosaic

API and Format

Adds an optional mosaic Cargo feature. No writer support or storage format change is introduced.

Documentation

No user-facing documentation is added in this PR. Predicate pushdown and async random-access reading are left as follow-up work.

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi merged commit 00a3660 into apache:main Jun 16, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants