Merge v1.5-variegata and fix main build#494
Draft
evertlammerts wants to merge 47 commits into
Draft
Conversation
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-05-12 05:54:24 - DuckDB SHA: daef7050ecc5252745dd35961d7923c1efc67204 - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/25715407807
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-05-13 16:12:26 - DuckDB SHA: 252f338be17cb7f2caade6a42cfa1b5fdb6e59e3 - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/25807759725
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-05-15 05:42:25 - DuckDB SHA: 6221d3e3f5d7ae0108cbef9eb9d4d01407315e3a - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/25901694091
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-05-18 06:23:46 - DuckDB SHA: b25ebaadb665ee38f74b5bbbb309fadc59e096d8 - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/26016352509
Fixes duckdb#460 A bug in the conjunction OR pushdown for pyarrow resulted in a too restrictive filter if one of the branches couldn't be translated.
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-05-19 06:22:27 - DuckDB SHA: 3a0db632336be3eb43f8a091a01d305b2d0cd046 - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/26079191264
…kdb#462) Related to duckdb#435 and duckdb#456 duckdb#435 tries to use the same connection across multiple threads, which is not supported. Specifically, in this case the result is overwritten and destroyed concurrently, causing a segfault. This PR fixes the segfault by synchronising access to the result with a recursive mutex. But note that connections cannot be used across threads, no matter what. duckdb#456: we sometimes need to destruct python-managed objects _after_ releasing the GIL. This PR makes sure that we hold on to the GIL during the destruction of PyRelation and Connection::Close(). Note: I rely on CI to check whether the fix actually works here...
Fixes duckdb#452 This makes sure that Polars' CSE wraps the lazyframe in a `CACHE[...]` in its plan. This way, the lazyframe will only be consumed once. We've seen this reported more often. The issue with joining a streaming result against itself is that a result can only be consumed once. One workaround is using multiple connections, but this only works if duckdb comes up with a query plan where a result is not referenced against itself. As it turns out, Polars allows I/O plugins to register with [`is_pure`](https://docs.pola.rs/api/python/dev/reference/api/polars.io.plugins.register_io_source.html), and Polars will de-duplicate them. Afaict, there's not really any downside. I don't see why anybody would want to treat a streaming result as not-pure, or what that would mean. I guess we'll see if that assumption will pass the test of time.
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-05-19 21:55:33 - DuckDB SHA: 14eca11bd9d4a0de2ea0f078be588a9c1c5b279c - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/26114996465
This PR allows creating relations using read only connections, i.e. `duckdb.connect(str(db_path), read_only=True)`.
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-05-28 06:08:41 - DuckDB SHA: 0bf2a03cb1973ac77de3a69c116c7b9f02f733ff - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/26557089319
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-05-29 06:30:32 - DuckDB SHA: 6f4fb97a10c65cce97f7c30d0bdc6844589b3b05 - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/26621310532
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-06-03 05:46:59 - DuckDB SHA: ac79ba69fe32d1bd612964daf08d3777837e331d - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/26865426747
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-06-08 05:44:12 - DuckDB SHA: a6fce56cf7d2e2c0ca7831b7d8db10413cdb269b - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/27117650655
Fixes duckdb#460 Remove dynamic predicates from the expression tree. Apart from it being a panic which is always a bad signal / bug, it's surprising that `pl.DataFrame.filter` cannot process a `pl.Expr` that polars itself created. In any case, this will remove dynamic predicates from the tree and either produce a pushdown filter fro duckdb, or filter in polars directly.
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-06-09 04:47:35 - DuckDB SHA: d9e204b353f12e38f36eed420de13e875b90af1d - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/27183709066
…d_parquet/from_parquet
Bump duckdb submodule: - Target branch: v1.5-variegata - Date: 2026-06-10 05:20:52 - DuckDB SHA: 894e3727d194d72295d10aa971798de10a82e657 - Trigger: https://github.com/duckdb/duckdb-python/actions/runs/27254183143
…d_parquet/from_parquet (duckdb#489) This PR makes the first arg of`read_parquet`/`from_parquet` go through `GetPathLike()` (like `read_csv`/`read_json`) so they accept `pathlib.Path`, generic `os.PathLike`, `bytes`, file-like objects, and mixed lists in addition to the existing `str`/`Sequence[str]`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.