[SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIELD_OUT_OF_BOUNDS by deepujain · Pull Request #54695 · apache/spark

deepujain · 2026-03-09T19:34:22Z

Summary

In Spark 4.0.0 ANSI mode is on by default. The JIRA asks to minimize suggestions that tell users to turn ANSI off. The error DATETIME_FIELD_OUT_OF_BOUNDS previously had a subclass WITH_SUGGESTION that appended "If necessary set spark.sql.ansi.enabled to false to bypass this error." This PR removes that suggestion; users can use try_* functions for safe behavior instead. The error now always uses WITHOUT_SUGGESTION (message is just the range description).

Change

error-conditions.json: Removed the WITH_SUGGESTION subclass from DATETIME_FIELD_OUT_OF_BOUNDS (kept only WITHOUT_SUGGESTION).
QueryExecutionErrors.scala: ansiDateTimeArgumentOutOfRange now throws DATETIME_FIELD_OUT_OF_BOUNDS.WITHOUT_SUGGESTION with only rangeMessage (removed ansiConfig parameter).
SQL test result files (date.sql.out, timestamp.sql.out, timestampNTZ/timestamp-ansi.sql.out, postgreSQL/date.sql.out, datetime-legacy.sql.out): Updated expected error class from WITH_SUGGESTION to WITHOUT_SUGGESTION and removed ansiConfig from expected messageParameters where the error is DATETIME_FIELD_OUT_OF_BOUNDS.
QueryExecutionAnsiErrorsSuite.scala: Added a regression test for DATETIME_FIELD_OUT_OF_BOUNDS so the error does not suggest disabling ANSI mode.

Tests

Added SPARK-49642: DATETIME_FIELD_OUT_OF_BOUNDS does not suggest ANSI config in QueryExecutionAnsiErrorsSuite.

Local validation:

./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642" passed.
./build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z time.sql" passed.

Fixes SPARK-49642

JIRA assignee for credit: deepujain

deepujain · 2026-03-12T04:44:53Z

Rebased on master. Local run: SQLQueryTestSuite for date/timestamp/datetime-legacy/timestamp-ansi (18 tests) and QueryExecutionAnsiErrorsSuite::SPARK-49642 (1 test)—all passed.

deepujain · 2026-03-15T15:17:06Z

Branch is rebased on master and local SQL/ANSI tests passed. No further changes from my side, ready for review when someone has bandwidth.

./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642"

deepujain · 2026-05-27T20:05:32Z

Rebased on current master. Ran ./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642" and the SQLQueryTestSuite date/timestamp/datetime-legacy/timestamp-ansi filters - passed. Ready for CI.

MaxGekk

1 blocking, 0 non-blocking, 0 nits. Correct approach, but the now-duplicate helper needs to be cleaned up before merging.

Design / architecture (1)

sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala:308: ansiDateTimeArgumentOutOfRangeWithoutSuggestion is now identical to this method — see inline

MaxGekk · 2026-06-10T11:52:04Z

-      messageParameters = Map(
-        "rangeMessage" -> e.getMessage,
-        "ansiConfig" -> toSQLConf(SQLConf.ANSI_ENABLED.key)),
+      errorClass = "DATETIME_FIELD_OUT_OF_BOUNDS.WITHOUT_SUGGESTION",


After this change, ansiDateTimeArgumentOutOfRangeWithoutSuggestion (lines 315–322) has an identical implementation to this method — same errorClass, same messageParameters, same context/summary/cause. The only remaining difference is the parameter type (Throwable vs Exception). The WithoutSuggestion suffix used to encode a real behavioral distinction; that distinction is gone now that both methods throw WITHOUT_SUGGESTION.

Suggested fix (self-contained in this PR):

Widen this method's parameter to Throwable (all callers pass DateTimeException/RuntimeException, compatible on both the Java and Scala sides).

Delete ansiDateTimeArgumentOutOfRangeWithoutSuggestion (lines 315–322).

Replace its 3 call sites in DateTimeUtils.scala (lines 940, 955, 957) with ansiDateTimeArgumentOutOfRange.

Addressed this by widening ansiDateTimeArgumentOutOfRange to Throwable, deleting ansiDateTimeArgumentOutOfRangeWithoutSuggestion, and switching the three DateTimeUtils call sites over. Rebased on current master and reran ./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642" plus ./build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z time.sql" - both passed.

MaxGekk

1 addressed, 0 remaining, 0 new.
0 blocking, 0 non-blocking, 0 nits.

The prior blocking finding (duplicate ansiDateTimeArgumentOutOfRange* helpers) is resolved — merged into a single ansiDateTimeArgumentOutOfRange(e: Throwable). Verified the behavior change reaches all callers: the make_date/make_timestamp sites in DateTimeExpressionUtils.java (which the PR doesn't touch) auto-rebind to the merged (Throwable) method and now emit WITHOUT_SUGGESTION — exactly what the updated golden files reflect. Golden-file coverage is complete, no imports are orphaned by dropping the toSQLConf(SQLConf...) call, QueryExecutionErrors is private[sql] (no public-API impact), and the new QueryExecutionAnsiErrorsSuite test covers the path end-to-end. LGTM.

PR description suggestions

The description says "No new tests added", but the diff adds the SPARK-49642: DATETIME_FIELD_OUT_OF_BOUNDS does not suggest ANSI config test to QueryExecutionAnsiErrorsSuite. The test is a good addition — just update that line of the description.

…D_OUT_OF_BOUNDS

deepujain · 2026-06-26T15:38:15Z

Rebased this on current master and updated the PR description to mention the new QueryExecutionAnsiErrorsSuite regression test. I reran ./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642" and ./build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z time.sql" locally; both passed. Ready for CI.

MaxGekk

All prior items resolved; 0 new. (0 blocking, 0 non-blocking, 0 nits.)

The earlier blocking finding (duplicate ansiDateTimeArgumentOutOfRange* helpers) is resolved — a single ansiDateTimeArgumentOutOfRange(Throwable) → WITHOUT_SUGGESTION, with the 3 DateTimeUtils sites rebinding to it.
My prior PR-description suggestion is addressed — the description now mentions the new QueryExecutionAnsiErrorsSuite regression test.

Re-verified the 9-file change on the rebased head: no source references to the removed DATETIME_FIELD_OUT_OF_BOUNDS.WITH_SUGGESTION remain, the error entry is well-formed (single WITHOUT_SUGGESTION subclass, sqlState 22023), the golden files are consistent, and the regression test asserts the message no longer contains spark.sql.ansi.enabled. Clean rebase (no scope pollution). LGTM.

MaxGekk · 2026-07-01T10:33:01Z

+1, LGTM. Merging to master/4.x.
Thank you, @deepujain.

…D_OUT_OF_BOUNDS ## Summary In Spark 4.0.0 ANSI mode is on by default. The JIRA asks to minimize suggestions that tell users to turn ANSI off. The error `DATETIME_FIELD_OUT_OF_BOUNDS` previously had a subclass `WITH_SUGGESTION` that appended "If necessary set spark.sql.ansi.enabled to false to bypass this error." This PR removes that suggestion; users can use `try_*` functions for safe behavior instead. The error now always uses `WITHOUT_SUGGESTION` (message is just the range description). ## Change - **error-conditions.json**: Removed the `WITH_SUGGESTION` subclass from `DATETIME_FIELD_OUT_OF_BOUNDS` (kept only `WITHOUT_SUGGESTION`). - **QueryExecutionErrors.scala**: `ansiDateTimeArgumentOutOfRange` now throws `DATETIME_FIELD_OUT_OF_BOUNDS.WITHOUT_SUGGESTION` with only `rangeMessage` (removed `ansiConfig` parameter). - **SQL test result files** (date.sql.out, timestamp.sql.out, timestampNTZ/timestamp-ansi.sql.out, postgreSQL/date.sql.out, datetime-legacy.sql.out): Updated expected error class from `WITH_SUGGESTION` to `WITHOUT_SUGGESTION` and removed `ansiConfig` from expected messageParameters where the error is `DATETIME_FIELD_OUT_OF_BOUNDS`. - **QueryExecutionAnsiErrorsSuite.scala**: Added a regression test for `DATETIME_FIELD_OUT_OF_BOUNDS` so the error does not suggest disabling ANSI mode. ## Tests Added `SPARK-49642: DATETIME_FIELD_OUT_OF_BOUNDS does not suggest ANSI config` in `QueryExecutionAnsiErrorsSuite`. Local validation: - `./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642"` passed. - `./build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z time.sql"` passed. Fixes SPARK-49642 **JIRA assignee for credit:** deepujain Closes #54695 from deepujain/SPARK-49642-remove-ansi-suggestion-datetime-out-of-bounds. Authored-by: Deepak Jain <deepujain@gmail.com> Signed-off-by: Max Gekk <max.gekk@gmail.com> (cherry picked from commit 890ce8d) Signed-off-by: Max Gekk <max.gekk@gmail.com>

MaxGekk · 2026-07-01T10:33:45Z

Merge summary (posted by merge_spark_pr.py):

merged into master 890ce8d6
merged into branch-4.x 565144b3

MaxGekk · 2026-07-01T10:34:52Z

@deepujain Congratulations with your first contribution to Apache Spark!

deepujain force-pushed the SPARK-49642-remove-ansi-suggestion-datetime-out-of-bounds branch 2 times, most recently from 96c0ccb to c298038 Compare March 12, 2026 04:44

deepujain force-pushed the SPARK-49642-remove-ansi-suggestion-datetime-out-of-bounds branch from c298038 to 8b93645 Compare March 15, 2026 15:15

deepujain force-pushed the SPARK-49642-remove-ansi-suggestion-datetime-out-of-bounds branch from 8b93645 to 351cbe0 Compare May 27, 2026 20:02

MaxGekk reviewed Jun 10, 2026

View reviewed changes

deepujain force-pushed the SPARK-49642-remove-ansi-suggestion-datetime-out-of-bounds branch from 351cbe0 to 39325ab Compare June 14, 2026 16:41

MaxGekk reviewed Jun 26, 2026

View reviewed changes

[SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIEL…

b3fc7a1

…D_OUT_OF_BOUNDS

deepujain force-pushed the SPARK-49642-remove-ansi-suggestion-datetime-out-of-bounds branch from 39325ab to b3fc7a1 Compare June 26, 2026 15:38

MaxGekk approved these changes Jul 1, 2026

View reviewed changes

MaxGekk changed the title ~~[SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIEL…~~ [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIELD_OUT_OF_BOUNDS Jul 1, 2026

MaxGekk closed this in 890ce8d Jul 1, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIELD_OUT_OF_BOUNDS#54695

[SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIELD_OUT_OF_BOUNDS#54695
deepujain wants to merge 1 commit into
apache:masterfrom
deepujain:SPARK-49642-remove-ansi-suggestion-datetime-out-of-bounds

deepujain commented Mar 9, 2026 •

edited

Loading

Uh oh!

deepujain commented Mar 12, 2026

Uh oh!

deepujain commented Mar 15, 2026

Uh oh!

deepujain commented May 27, 2026

Uh oh!

MaxGekk left a comment

Uh oh!

MaxGekk Jun 10, 2026

Uh oh!

deepujain Jun 14, 2026

Uh oh!

MaxGekk left a comment

Uh oh!

deepujain commented Jun 26, 2026

Uh oh!

MaxGekk left a comment

Uh oh!

MaxGekk commented Jul 1, 2026

Uh oh!

MaxGekk commented Jul 1, 2026

Uh oh!

MaxGekk commented Jul 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

deepujain commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Change

Tests

Uh oh!

deepujain commented Mar 12, 2026

Uh oh!

deepujain commented Mar 15, 2026

Uh oh!

deepujain commented May 27, 2026

Uh oh!

MaxGekk left a comment

Choose a reason for hiding this comment

Design / architecture (1)

Uh oh!

MaxGekk Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

deepujain Jun 14, 2026

Choose a reason for hiding this comment

Uh oh!

MaxGekk left a comment

Choose a reason for hiding this comment

PR description suggestions

Uh oh!

deepujain commented Jun 26, 2026

Uh oh!

MaxGekk left a comment

Choose a reason for hiding this comment

Uh oh!

MaxGekk commented Jul 1, 2026

Uh oh!

MaxGekk commented Jul 1, 2026

Uh oh!

MaxGekk commented Jul 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

deepujain commented Mar 9, 2026 •

edited

Loading