docs: user guide for the substrait.dataframe API (Zensical)#214
Open
nielspardon wants to merge 6 commits into
Open
docs: user guide for the substrait.dataframe API (Zensical)#214nielspardon wants to merge 6 commits into
nielspardon wants to merge 6 commits into
Conversation
Adds a multi-page guide for the ergonomic DataFrame/Expr API (from substrait-io#204), built with Zensical + mkdocstrings, plus a README introduction pointing to it. - docs/: 16-page guide (data sources, types, expressions, the `f` namespace, transformations, joins, aggregations, set ops, window functions, subqueries, parameters/context, DDL & writes, custom extensions, consuming plans) and a mkdocstrings-generated API reference. Every code example is verified against the real API. - zensical.toml: Material theme, nav, and the mkdocstrings Python handler pointed at src/. - pyproject.toml: `docs` dependency group (zensical, mkdocstrings-python) and a pixi `docs` environment with `docs-build`/`docs-serve` tasks. - .github/workflows/docs.yml: build-check on PRs (no deploy; a ready-to-enable GitHub Pages deploy job is included as a comment). - README.md: new "Building plans with the DataFrame API" section; the raw-proto examples are reframed as the low-level API. 🤖 Generated with AI
The substrait.dataframe docstrings used reStructuredText constructs that render as literal text under mkdocstrings' Markdown renderer. Convert them so the generated API reference reads well: - RST literal blocks (`::` + indented code) -> fenced ```python blocks. - Inline roles (:class:/:meth:/:func:/:mod:`...`) -> backticked names. - Fix two illustrative examples that would not resolve (`f.substring(..., 1, 3)` -> `f.upper(...)`; substring needs i32 offsets). Docstrings only; no behavior change. The 295 dataframe tests still pass and the 93 verified guide snippets are unaffected. 🤖 Generated with AI
Adds multi-version documentation publishing using the Zensical-compatible fork of mike, which commits each version to the gh-pages branch as a subdirectory of site_url. - zensical.toml: set site_url (required for versioning) and add [project.extra.version] (provider = "mike") to render the version selector. - .github/workflows/docs-deploy.yml: deploy job that runs on release tags (v*.*.*) and via workflow_dispatch. The manual trigger bootstraps the gh-pages branch so Pages can be pointed at it; tag pushes then publish each release as its minor series with a `latest` alias + root redirect. mike is installed ad-hoc (git-only), so uv.lock is untouched. - docs.yml: the PR build-check now points at docs-deploy.yml for publishing instead of the old single-version artifact-deploy note. Enabling it needs a one-time manual run (creates gh-pages) plus flipping Settings -> Pages -> Source to the gh-pages branch. 🤖 Generated with AI
Add a Documentation section covering local preview/build (pixi run docs-serve / docs-build), the Markdown-over-RST docstring convention, and where docs are published. 🤖 Generated with AI
The setup step referenced 'uv sync --extra test', but there is no 'test' extra (it errors). Test dependencies live in the default 'dev' dependency group, so plain 'uv sync' is the correct setup command. 🤖 Generated with AI
Keep the workflow header to a concise description of what it does; the Pages/gh-pages bootstrap runbook doesn't belong in the workflow file. 🤖 Generated with AI
nielspardon
requested review from
andrew-coleman,
benbellick,
bestbeforetoday,
tokoko and
vbarua
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds user-facing documentation for the ergonomic native DataFrame/Expr API
(
substrait.dataframe) introduced in #204, plus a README introduction pointingto it. There was previously no guide for the new API — the README still led with
the verbose raw-
protoconstruction.Built with Zensical (Material-style,
zensical.toml) andmkdocstrings for the generated API reference.
Contents
docs/— a multi-page guide: overview, getting started, data sources,types & schemas, expressions, the
ffunction namespace, transformations,joins, aggregations, set operations, window functions, subqueries, parameters
& context, DDL & writes, custom extensions, consuming plans, and a
mkdocstrings-generated API reference. Every code example is verified to build
a valid plan against the real API.
zensical.toml— theme, nav,site_url, the mkdocstrings Python handler(pointed at
src/), and themikeversion provider for multi-version docs.pyproject.toml— adocsdependency group (zensical,mkdocstrings-python) plus a pixidocsenvironment withdocs-build/docs-servetasks..github/workflows/docs.ymlruns a build-check on every PR..github/workflows/docs-deploy.ymlpublishes versioned docs to thegh-pagesbranch viamike, on release tags (v*.*.*) and viaworkflow_dispatch.README.md— new "Building plans with the DataFrame API" section; theraw-
protoexamples are reframed as the low-level API.CONTRIBUTING.md— a Documentation section (local preview/build,docstring convention); also corrects the dev-setup command (
uv sync, not thenon-existent
--extra test).substrait.dataframedocstrings fromreStructuredText (
::literal blocks,:class:/:meth:roles) to Markdown sothe generated reference renders cleanly. Docstrings only; no behavior change.
Building locally
pixi run docs-serve # or: uv run --group docs zensical serveEnabling GitHub Pages (one-time, after merge)
Versioned publishing needs the
gh-pagesbranch to exist before Pages can pointat it:
gh-pages).gh-pages/ (root).v*.*.*release tag publishes that minor series andupdates the
latestalias. Adjustsite_urlinzensical.tomlif the siteis served from a custom domain.
Verification
zensical buildcompletes with no issues (includes link validation).tests/dataframetests pass; 93 guide code snippets each build a valid plan.🤖 Generated with AI