Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/workflows/_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,18 @@ jobs:
- run: uv python pin ${{ matrix.python-version }}
- run: just install
- run: just test --cov-report xml

pytest-freethreaded:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: extractions/setup-just@v4
- uses: astral-sh/setup-uv@v8.2.0
with:
enable-cache: true
cache-dependency-glob: "**/pyproject.toml"
- run: uv python install 3.14t
- run: uv python pin 3.14t
- run: just install
- run: uv run --no-sync python -c "import sys; assert not sys._is_gil_enabled(), 'GIL is enabled'"
- run: just test --cov-report xml
4 changes: 4 additions & 0 deletions architecture/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@

`httpx2` is part of the public surface. Exposing `httpx2.Request`/`httpx2.Response` is the design — `httpware` does not own a full abstraction over the underlying HTTP client.

## Supported versions

`httpware` requires Python 3.11+ and is tested on the standard (GIL) builds of 3.11–3.14 in CI. It also carries Beta-tier free-threading support (`Free Threading :: 2 - Beta` classifier): a dedicated `pytest-freethreaded` CI job runs the full suite on free-threaded CPython `3.14t` with the GIL disabled. `3.13t` is deferred pending an msgspec cp313t wheel (see `planning/deferred.md`); see [Testing](testing.md) for the `stress` marker convention and [Resilience](resilience.md) for what free-threading correctness covers.

## Architectural invariants

These are non-negotiable, but **enforcement varies — do not assume CI will catch a violation.** Machine-checked: `print()` (ruff `T201`) and a blanket `# type: ignore` (ruff `PGH003`). Partially checked: the `httpx2._` ban — ruff `SLF001` flags private *attribute* access (`httpx2._foo`) but not a *used* private import (`from httpx2._internal import …`). Review-only: the future-import and global-logging bans, and `# type: ignore[<code>]` vs `# ty: ignore[<code>]`. The "why" exists so future contributors can judge edge cases instead of blindly following the rule.
Expand Down
2 changes: 2 additions & 0 deletions architecture/resilience.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

`httpware` ships a resilience suite under `httpware.middleware.resilience`, composed via the standard middleware chain (Seam A). It is pure stdlib — no optional extra.

The thread-shared components — the sync `Bulkhead` and `CircuitBreaker` (`threading.Semaphore`/`threading.Lock`) and the `RetryBudget` shared across a sync `Client`/`AsyncClient` pair — are stress-verified for correctness under free-threaded CPython (3.14t, GIL disabled) via real-thread `pytest.mark.stress` tests; the `httpx2` connection pool httpware's terminal relies on has its own boundary stress test (see [Testing](testing.md)). The async `AsyncBulkhead`/`AsyncCircuitBreaker` hold no thread-shared mutable state — they use asyncio primitives and the single-event-loop guard rejects cross-loop use (deterministically tested, see [Bulkhead](#bulkhead)) — so they are single-loop-bound and not thread-parallel-stressed by design. This is a correctness certification (`Free Threading :: 2 - Beta` classifier), not a performance claim — a contention benchmark on `RetryBudget` found free-threaded 3.14t ~1.9x *slower* than GIL 3.11 for a single-shared-lock hot loop (`planning/audits/2026-07-18-free-threading-audit.md`); lock contention dominates that access pattern.

## Retry + RetryBudget

`Retry` (and `AsyncRetry`) is a retry middleware backed by a Finagle-style `RetryBudget` — a token bucket that caps the proportion of traffic spent on retries so a degraded backend cannot be amplified into a retry storm. `RetryBudget` is a single thread-safe class shared by both worlds: all mutations go through a `threading.Lock`, so state is never torn. "Safe" here means no corruption, not non-blocking — when one budget is shared across a (sync `Client`, `AsyncClient`) pair, a sync thread holding the lock can briefly block the event-loop thread's acquisition. The critical section is intentionally tiny to bound that latency. Backoff between attempts uses full-jitter.
Expand Down
3 changes: 2 additions & 1 deletion architecture/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@
- **`pytest-asyncio` auto mode.** Async test functions do not require `@pytest.mark.asyncio`. The setting lives in `pyproject.toml` under `[tool.pytest.ini_options]`.
- **`httpx2.MockTransport` for transport mocking, not `respx`.** Tests construct `httpx2.AsyncClient(transport=httpx2.MockTransport(handler))` and pass it as `httpx2_client=` to `AsyncClient` (or the sync equivalent `httpx2.Client(transport=httpx2.MockTransport(handler))` to `Client`). `MockTransport` is a first-party `httpx2` API; `respx` targets `httpx` (not `httpx2`) and patches its internals directly — see `docs/testing.md` for why.
- **Hypothesis property-based tests** for concurrency-sensitive code: `RetryBudget`, `Bulkhead`, retry interleaving. Files are named `test_*_props.py` so they are easy to grep and treat separately in CI.
- **Performance tests are opt-in.** The `perf` pytest marker is registered in `pyproject.toml`; the default `addopts` line includes `-m 'not perf'`. Run benchmarks explicitly with `pytest -m perf`.
- **Benchmarks are standalone scripts, not pytest tests.** Performance characterization (e.g. `benchmarks/contention.py`) lives under `benchmarks/` (coverage-omitted via `run.omit`), run explicitly and never gated in CI — shared-runner perf is too noisy to gate on.
- **`stress` marker for free-threading proof.** Registered in `pyproject.toml`, `stress`-marked tests drive real thread parallelism (`ThreadPoolExecutor`, shared instances, high iteration counts, invariant-not-timing assertions) against `httpware`'s Lock/Semaphore-based components and the shared `httpx2` connection pool. They run in the default suite too, so they count toward coverage under the GIL — but the GIL serializes bytecode execution, so a GIL run only exercises the code path, it does not *prove* thread-safety. That proof comes from the `pytest-freethreaded` CI job, which runs the full suite on free-threaded CPython `3.14t` with the GIL disabled (the job asserts `sys._is_gil_enabled()` is False in a shell step before running the suite). `3.13t` is deferred — see `planning/deferred.md` — pending an msgspec cp313t wheel.
- **Coverage is 100% line coverage.** New code is expected to maintain this.
Empty file added benchmarks/__init__.py
Empty file.
54 changes: 54 additions & 0 deletions benchmarks/contention.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
"""Contention benchmark: shared Lock-based resilience components, GIL vs free-threaded.

Answers whether removing the GIL helps httpware's shared components or whether lock contention
eats the gain. Run under both interpreters and compare:

uv run --python 3.11 benchmarks/contention.py
uv run --python 3.14t benchmarks/contention.py

Not a test and not a CI gate (shared-runner perf is too noisy); a characterization tool with a
recorded baseline in planning/audits/2026-07-18-free-threading-audit.md.
"""

import sys
import threading
import time

from httpware.middleware.resilience.budget import RetryBudget


_N_THREADS = 8
_N_OPS = 200_000


def _run() -> float:
budget = RetryBudget(ttl=60.0, min_retries_per_sec=1_000_000.0, percent_can_retry=1.0)

def worker() -> None:
for _ in range(_N_OPS):
budget.deposit()
budget.try_withdraw()

threads = [threading.Thread(target=worker) for _ in range(_N_THREADS)]
start = time.perf_counter()
for t in threads:
t.start()
for t in threads:
t.join()
return time.perf_counter() - start


def main() -> None:
gil = getattr(sys, "_is_gil_enabled", lambda: True)()
elapsed = _run()
ops = _N_THREADS * _N_OPS * 2
message = (
f"python={sys.version.split()[0]} gil_enabled={gil} threads={_N_THREADS} "
f"ops={ops} elapsed={elapsed:.3f}s throughput={ops / elapsed:,.0f} ops/s"
)
# T201 is fine here: benchmarks/ is not library code and is coverage-omitted.
print(message) # noqa: T201


if __name__ == "__main__":
main()
90 changes: 90 additions & 0 deletions planning/audits/2026-07-18-free-threading-audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# httpware free-threading (nogil) audit — 2026-07-18

**Status:** complete
**Scope:** empirical findings backing `planning/changes/2026-07-18.01-nogil-free-threading-support.md`
— the extras wheel matrix, the httpx2 shared-pool boundary result, the
free-threaded stress-test suite, and the contention benchmark baseline
(`benchmarks/contention.py`).

## Extras wheel matrix (free-threaded interpreters)

| Extra | 3.13t | 3.14t |
|---|---|---|
| pydantic | ✓ | ✓ |
| msgspec | ✗ (no cp313t wheel, versions 0.18–0.21.1) | ✓ |
| otel (opentelemetry) | ✓ (pure-Python, no wheel gap) | ✓ |

3.14t is the only free-threaded interpreter with complete extras coverage
from prebuilt wheels; all three extras keep the GIL disabled after import
(verified — none silently re-enables it). This is why free-threaded CI
(`.github/workflows/_checks.yml`, `pytest-freethreaded` job) targets `3.14t`
only; 3.13t is deferred in `planning/deferred.md` until msgspec ships a
cp313t wheel.

## httpx2 shared-pool boundary

`tests/test_httpx2_freethreaded_boundary.py::test_httpx2_shared_pool_no_crosstalk_under_parallelism`
(marked `stress`) drives 16 threads sharing one `httpx2` client + connection
pool, 100 requests per thread (1,600 total), in a single run on 3.14t with the
GIL disabled. Every response is verified against its own request. Result:
**zero cross-talk, zero crashes.** (The initial exploratory validation during
design was heavier — 32 threads, 3 × 12,800 requests — and also clean; the
committed test is the trimmed, CI-fast regression guard.) httpx2 is a
dependency httpware can't self-certify beyond this boundary test — this is the
recorded regression evidence that it holds under true thread parallelism.

## Free-threaded stress-test suite

Real thread-parallelism tests exercise httpware's Lock/Semaphore-based
components; five carry `pytest.mark.stress` and a sixth deterministically
covers the cross-loop guard. All pass on 3.14t with `sys._is_gil_enabled() is
False` (verified directly, and via the `pytest-freethreaded` CI job which
asserts the GIL is disabled before running the suite):

- `tests/test_retry_budget_threadsafety.py::test_concurrent_deposit_withdraw_does_not_corrupt` (`stress`)
- `tests/test_retry_budget_threadsafety.py::test_concurrent_only_deposit_count_matches` (`stress`)
- `tests/test_circuit_breaker_freethreaded_stress.py::test_circuit_breaker_opens_consistently_under_parallel_failures` (`stress`)
- `tests/test_bulkhead_freethreaded_stress.py::test_bulkhead_never_exceeds_max_concurrent_under_parallelism` (`stress`)
- `tests/test_httpx2_freethreaded_boundary.py::test_httpx2_shared_pool_no_crosstalk_under_parallelism` (`stress`)
- `tests/test_bulkhead.py::test_cross_loop_acquire_raises_runtimeerror` (deterministic;
covers the guard's outer cross-loop raise. The inner double-checked-lock arm
stays `# pragma: no cover` — free-threading makes it reachable but only
nondeterministically, so it is not asserted)

Each stress test uses invariant assertions (final counts, exhaustion bounds,
state consistency) rather than interleaving-dependent timing, per
`architecture/testing.md`'s stress-test convention.

## Contention benchmark: GIL vs free-threaded

`benchmarks/contention.py` (committed, not a CI gate — shared-runner perf is
too noisy to gate on) drives 8 threads × 200,000 iterations of
`RetryBudget.deposit()` + `RetryBudget.try_withdraw()` against one shared
`RetryBudget` (3,200,000 total lock-guarded ops), measuring wall-clock
throughput. Run twice per interpreter for stability:

```
uv run --no-sync python benchmarks/contention.py
python=3.11.9 gil_enabled=True threads=8 ops=3200000 elapsed=0.869s throughput=3,684,320 ops/s
python=3.11.9 gil_enabled=True threads=8 ops=3200000 elapsed=0.869s throughput=3,683,289 ops/s

<ft-run venv>/bin/python benchmarks/contention.py
python=3.14.6 gil_enabled=False threads=8 ops=3200000 elapsed=1.626s throughput=1,967,685 ops/s
python=3.14.6 gil_enabled=False threads=8 ops=3200000 elapsed=1.634s throughput=1,958,734 ops/s
```

**Interpretation:** for this workload — a tiny critical section
(`deque` purge + append/compare under `threading.Lock`) hammered by 8 threads
with no non-lock work between acquisitions — free-threaded 3.14t is
**~1.9x slower** than GIL 3.11 (≈1.96M ops/s vs ≈3.68M ops/s), so **lock
contention dominates and nogil does not help here**: free threads spend more
wall-clock time contending for the same `threading.Lock` than the GIL's
cooperative bytecode-level serialization costs, and `RetryBudget`'s
correctness (proven by the stress test above) does not translate into a
throughput win under this access pattern. This is expected for a
single-shared-lock hot loop and is not a regression to fix — a real client
workload interleaves this critical section with I/O and non-lock CPU work
where free-threading's benefit (true parallel non-lock work) would show up
where this microbenchmark cannot show it. `RetryBudget`'s thread-safety
contract (`architecture/resilience.md`) is unaffected either way; this
benchmark characterizes throughput, not correctness.
95 changes: 95 additions & 0 deletions planning/changes/2026-07-18.01-nogil-free-threading-support.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
summary: Certified httpware under free-threaded CPython (PEP 703) — a `pytest-freethreaded` CI job on 3.14t, real-thread stress tests for RetryBudget, CircuitBreaker, Bulkhead, and the httpx2 shared-pool boundary, plus a deterministic event-loop-guard cross-loop test, a committed contention benchmark (nogil ~1.9x slower for the single-shared-lock workload — correctness, not speedup), and the `Free Threading :: 2 - Beta` classifier shipped last. 3.13t deferred on the msgspec cp313t wheel gap.
---

# Design: Free-threading (nogil) support

## Summary

Certify that httpware runs correctly under free-threaded CPython (PEP 703),
backed by evidence rather than a bare classifier. Adds a free-threaded CI job
(3.14t, full suite + all extras), parallel stress tests for the Lock-based
resilience components and the shared httpx2 connection pool, a committed
contention benchmark with a recorded baseline, and — last, once everything is
green — the `Programming Language :: Python :: Free Threading` classifier.
Verify-first: the public claim trails the proof.

## Motivation

httpware is a concurrency library: its resilience suite (`RetryBudget`,
`CircuitBreaker`, `Bulkhead`) is built on `threading.Lock`/`Semaphore` and
documents thread-safety invariants, and a shared client across threads is a
documented usage pattern. Free-threaded CPython (free-threaded builds available
since 3.13, officially supported in 3.14) removes the GIL that currently masks
any latent races. The library already ships 3.13/3.14 classifiers and tests
those *GIL* builds, but nothing exercises true thread parallelism.

Two empirical findings (2026-07-18) de-risk the work:

- **httpx2 2.5.0 holds under free-threading.** 32 threads sharing one client +
connection pool, 3 × 12,800 requests on 3.14t with the GIL disabled, each
response verified against its request: zero cross-talk, zero crashes.
- **Extras wheel matrix on free-threaded interpreters.** pydantic ✓ (3.13t +
3.14t), msgspec ✓ 3.14t but ✗ no cp313t wheel (0.18–0.21.1), otel pure-Python
✓. **3.14t is the only interpreter with complete extras coverage from
prebuilt wheels;** all three keep the GIL disabled after import.

## Design

Five workstreams, landed verify-first (classifier last):

1. **CI enablement.** Add one free-threaded job on `3.14t` to
`.github/workflows/_checks.yml`, running the full suite + all extras (uv
installs `3.14t`; extras resolve from prebuilt free-threaded wheels). The
existing GIL matrix (3.11–3.14) is unchanged. 3.13t-free-threaded is deferred
until msgspec ships a cp313t wheel (record in `deferred.md`).
2. **Free-threaded audit + stress tests.** Audit httpware's own lock usage under
parallelism, then add stress tests (real threads via `ThreadPoolExecutor`,
shared instances, high iteration counts, invariant assertions) for
`RetryBudget` token accounting, `CircuitBreaker` transitions, `Bulkhead`
semaphore counting, and the event-loop guard's cross-loop rejection. TDD per
component. **Specific target:** `_event_loop_guard.py`'s `# pragma: no cover`
double-checked-locking arm is annotated "single-threaded tests can't
trigger" — free-threading makes it reachable, so the pragma is re-examined
(covered → removed, or reworded).
3. **httpx2 boundary test.** Promote the shared-pool cross-talk stress harness
into the suite (marked) as living regression evidence for the one dependency
httpware can't self-certify.
4. **Benchmarks.** A committed `benchmarks/` script measuring contended
throughput (GIL vs free-threaded) on the shared Lock components — does nogil
help, or does lock contention eat the gain? Baseline numbers recorded under
`planning/audits/`.
5. **Certify + promote.** Add the `Free Threading` classifier; promote into
`architecture/resilience.md` (thread-safety prose), `architecture/testing.md`
(stress-test convention + marker), `architecture/overview.md`; release notes.

Stress tests carry a `stress` pytest marker so the free-threaded job runs them;
they exercise concurrency under the GIL too but only *prove* safety on 3.14t.

## Non-goals

- **3.13t support** — blocked on a msgspec cp313t wheel; deferred, not designed out.
- **Certifying httpx2 itself** — out of our control; we ship evidence, optionally upstream later.
- **CI perf regression gates** — benchmarks are characterization (committed script + recorded baseline), not a gate; shared-runner perf is too noisy.
- **Reworking the Lock-based design** — the audit validates it; it does not rewrite it.

## Testing

- Full suite green under `uv run --python 3.14t` locally and in the new CI job,
GIL disabled (the `pytest-freethreaded` job asserts `sys._is_gil_enabled() is
False` in a shell step before running the suite).
- `pytest -m stress` on 3.14t: RetryBudget / breaker / bulkhead invariants hold
across many-thread runs, and the httpx2 boundary test shows no cross-talk; a
separate deterministic test covers the event-loop guard's cross-loop rejection.
- `just lint-ci` and the planning validator unchanged and green.

## Risk

- **msgspec 3.13t wheel gap (med likelihood / low impact).** Scopes free-threaded
CI to 3.14t only. Mitigation: documented deferral; revisit on wheel availability.
- **Flaky stress tests (med / med).** Probabilistic race detection. Mitigation:
invariant assertions (not interleaving-dependent), tuned iteration counts, the
`stress` marker isolates them from the default run.
- **Audit surfaces a real race (low / high).** The 2026-06-14 deep audit already
reasoned the loop-guard fast path is benign; a real race would be a genuine bug
caught *before* shipping the claim — which is the point of verify-first.
4 changes: 4 additions & 0 deletions planning/deferred.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ As of 0.7.0, all planned epics (3, 4, 5, 6) are closed — see the [change Index

## Open

### Free-threading

- **3.13t free-threaded CI** — blocked on a msgspec cp313t wheel; add the interpreter to the `pytest-freethreaded` job when the wheel ships.

### Resilience

- **CircuitBreaker — manual control** (`src/httpware/middleware/resilience/circuit_breaker.py`) — the trip-mode work is done (0.13.0 shipped the opt-in time-based failure-rate mode) and 0.14.0 shipped the read-only `state` property + public `CircuitState` enum. The one remaining piece is `force_open`/`force_closed` (Polly's `ManualControl`) — the genuinely YAGNI half for an HTTP *client* (you'd usually just stop sending requests), keyed off the 0.10.0 audit's events-only control-surface decision (decision 4). Demand-gated.
Expand Down
Loading
Loading