fix(isthmus): elide redundant identity projections#1009
Open
nielspardon wants to merge 1 commit into
Open
Conversation
Calcite wraps queries such as `SELECT "a", "B" FROM foo GROUP BY a, b` in a name-changing identity projection (`LogicalProject(a=[$0], B=[$1])`) whose field references are `$0, $1` in order — only the output names differ. `SubstraitRelVisitor.visit(Project)` turned every such projection into a Substrait `Project`, but Substrait carries output names on `Plan.Root`, not on the `Project`, so the node is pure redundancy. The full round trip failed because the reverse conversion elides the identity projection, leaving the two sides structurally different. Skip emitting a `Project` for an identity projection (all input refs, in order, with matching types, as reported by `RexUtil.isIdentity`). The check runs on both conversion legs, so both sides converge to the project-free form. Output names are unaffected — `convert(RelRoot, ...)` takes them from `validatedRowType`. This lets the identity-projection test workaround be removed: the DDL/DML round trips now pass the stronger plain `assertFullRoundTrip`. Closes substrait-io#371 🤖 Generated with AI
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Calcite wraps queries such as
SELECT "a", "B" FROM foo GROUP BY a, bin a name-changing identity projection:The field references are
$0, $1in order — only the output names differ (b→B).SubstraitRelVisitor.visit(Project)turned every such projection into a SubstraitProject, but Substrait carries output names onPlan.Root, not on theProject, so the node is pure redundancy.assertFullRoundTripfailed because the reverse conversion (Substrait → Calcite → Substrait) elides the identity projection, soroot1kept the project whileroot3did not:Change
Skip emitting a
Projectfor an identity projection — all expressions are input refs, in order, with matching types, as reported by Calcite's ownRexUtil.isIdentity. Because the check lives invisit(Project), it applies to both conversion legs, so the two sides converge to the project-free form and round-trip cleanly. Output names are unaffected:convert(RelRoot, ...)derives them fromvalidatedRowType, independent of the projection.Scope is limited to identity projections. Non-identity all-field-ref projections (permutations / column pruning) still emit a
Projectas before; that optimization is noted as future work in the retained TODO.Tests
NameRoundtripTest.roundTripNameChangingProjectionOverAggregatecovering the exact query from the issue.assertFullRoundTripWithIdentityProjectionWorkaroundhelper (whose Javadoc noted it should be removed long-term) is deleted;DdlRoundtripTestandDmlRoundtripTestnow use the stronger plainassertFullRoundTrip.SubqueryPlanTest.existsNestedCorrelatedSubquerynavigation updated: the innerUNIQUE (SELECT * ...)identity projection is now elided, matching how the outerEXISTSsubquery was already represented. Downstream field-reference / steps-out assertions are unchanged, confirming the elision is semantically neutral.:isthmus:test,:isthmus:check(PMD + javadoc), andspotlessCheckpass.Closes #371
🤖 Generated with AI