build(kernel): add optional [kernel] extra for use_kernel=True

[vikrantpuppala]

What

databricks-sql-kernel is now published to PyPI, so the Rust kernel backend (use_kernel=True) can ship as an optional dependency instead of being a local-dev-only build. This adds the [kernel] extra:

pip install "databricks-sql-connector[kernel]"

sql.connect(server_hostname=host, http_path=http_path, access_token=token, use_kernel=True)

Changes

• pyproject.toml
  • Declare databricks-sql-kernel = {version = "^0.1.0", optional = true, python = ">=3.10"}. Gated to Python ≥ 3.10 because the wheel is cp310-abi3 (Requires-Python: >=3.10). The connector's own floor (3.8) is unchanged; on 3.8/3.9 the extra resolves to nothing.
  • Add kernel = ["databricks-sql-kernel", "pyarrow"]. pyarrow is listed explicitly: the kernel result path (backend/kernel/result_set.py) imports it unconditionally to wrap the Arrow batches the kernel returns. It's already pulled transitively via the kernel wheel's pyarrow>=23.0.1,<24, but naming it documents the connector-side requirement and lets pip co-resolve both constraints at install time.
• backend/kernel/_errors.py — the use_kernel=True ImportError now points at pip install "databricks-sql-connector[kernel]" and notes Python ≥ 3.10 (was the obsolete "not yet published, build locally" hint).
• README.md — document the extra, usage, and the Python ≥ 3.10 / pyarrow notes.

How it works

use_kernel=True imports databricks_sql_kernel at backend load. With the extra → import succeeds, kernel path is live. Without it → friendly ImportError telling the user to install [kernel]. The kernel stays a soft dependency: default installs don't pull the Rust wheel.

Local verification

Kernel served from a locally-built cp310-abi3 wheel (the published package isn't yet mirrored on the dev PyPI proxy I have access to):

• pip install "databricks-sql-connector[kernel]" → connector + databricks-sql-kernel 0.1.2 + pyarrow 23.0.1 all install; use_kernel=True runs a live query end-to-end (backend KernelDatabricksClient, result 42).
• Plain pip install databricks-sql-connector → use_kernel=True raises the friendly ImportError (mentions [kernel] + Python ≥ 3.10).
• Wheel tag confirmed cp310-abi3, matching the python>=3.10 marker.

⚠️ Before merging

1. poetry lock — poetry.lock must be regenerated with the databricks-sql-kernel entry. Not included here: it requires the kernel to be resolvable on the index poetry/CI use (the JFrog db-pypi proxy). Run it once the package resolves there.
2. Confirm index reachability — verify databricks-sql-kernel resolves on db-pypi (mirrors public PyPI). If the publish hasn't propagated, declaring the dep will break poetry lock for every CI job. (I could not verify this from my environment — public PyPI egress is blocked and JFrog needs CI's OIDC token.)
3. CI smoke test — consider adding a pip install ".[kernel]" job (or extending kernel-e2e.yml, which currently source-builds at KERNEL_REV) so the published-wheel install path is exercised.

Opened as a draft pending items 1–2.