Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions architecture/concurrency.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,12 @@ resolve from it):
with `use_lock=False` opt out of the lock and are single-thread-only.
- **Registry memoization is lock-free and idempotent.** The compiled resolver, the
wiring plan, and their registry caches (`_resolvers`, `_plans`) are pure functions
of `(provider, registry version)`. Two threads racing to build the same entry
produce identical objects and store the same `(version, object)` tuple; the worst
case is one duplicated build, never a wrong result. The cycle-guard `_building`
of `(provider, registry contents)`, cleared on mutation. Two threads racing to build
the same entry produce identical objects; the worst
case is one duplicated build, never a wrong result. Clearing on mutation is sound because
mutation is a single-threaded configure-phase operation (see [The lifecycle](#the-lifecycle)
above); a program that mutates the registry during concurrent resolution loses the old
version-stamp approach's rebuild-stale safety net. The cycle-guard `_building`
set is **thread-local**: it tracks which providers are being compiled on *this*
call stack, so a genuine same-thread `A -> B -> A` cycle is still caught by the
back-edge thunk, while a concurrent first-resolve of the same provider on another
Expand Down
4 changes: 2 additions & 2 deletions architecture/containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ The four registries split into two categories:

| Registry | Shared across container tree? | Purpose |
|---|---|---|
| `ProvidersRegistry` | Yes — all containers share one instance | Maps `type → AbstractProvider`; populated at root construction time from `groups`, and later via `Container.add_providers`. Also holds the shared `_plans` wiring-plan memo (keyed by `provider_id`, stamped with `version`), so a plan is built once tree-wide. |
| `ProvidersRegistry` | Yes — all containers share one instance | Maps `type → AbstractProvider`; populated at root construction time from `groups`, and later via `Container.add_providers`. Also holds the shared `_plans` wiring-plan memo (keyed by `provider_id`, cleared on registry mutation), so a plan is built once tree-wide. |
| `OverridesRegistry` | Yes — all containers share one instance | Maps `provider_id → override object`; used by tests to substitute real instances. |
| `CacheRegistry` | No — each container has its own | Maps `provider_id → CacheItem`; stores resolved singleton instances and their finalizers for this scope level. |
| `ContextRegistry` | No — each container has its own | Maps `type → runtime object`; populated via `context=` at construction or `container.set_context()` after the fact. |
Expand All @@ -93,7 +93,7 @@ registry it mutates is shared tree-wide. `Container` tracks a private
after registering and, if that fails, removes the just-added batch again —
the container ends up either fully registered and valid, or unchanged.
Whether the graph is *currently* validation-clean is tracked separately and
registry-level, by `ProvidersRegistry.validated_version` (see
registry-level, by `ProvidersRegistry`'s `_validated` flag (see
[validation.md](validation.md#what-validate-checks)). Because that registry is
shared tree-wide, validating any container in the tree marks the whole graph
clean, so a child's `resolve` benefits from the root's validation — its runtime
Expand Down
4 changes: 2 additions & 2 deletions architecture/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ When a `Factory` is resolved, `WiringPlan.build` (in `modern_di/wiring.py`) iter
partitions it into the plan; the factory's **compiled resolver** then invokes each matched dependency's own
compiled resolver, captured by reference at compile time (the chain of closure calls that replaces per-edge
`resolve_provider` recursion — see [resolution.md](resolution.md#compiled-resolvers)). The plan is memoized on the
`ProvidersRegistry` (keyed by `provider_id`, stamped with the registry version) and rebuilt when that version
changes (i.e. after `add_providers`); the compiled resolver is memoized the same way. Resolution errors are
`ProvidersRegistry` (keyed by `provider_id`) and cleared on registry mutation
(i.e. after `add_providers`); the compiled resolver is memoized the same way. Resolution errors are
annotated with a breadcrumb describing the current factory, so the full chain appears in the exception.

### Static kwargs and `skip_creator_parsing`
Expand Down
22 changes: 11 additions & 11 deletions architecture/resolution.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ memoized. There is no interpreted fallback in the shipped tree; every provider t
## Compiled resolvers

`ProvidersRegistry.resolver_for(provider)` returns the provider's compiled resolver — a `Callable[[Container], T]`
built on first request by `resolver_compiler.compile_resolver` and then **memoized**, keyed by `provider_id` and
stamped with the `ProvidersRegistry.version` it was built against, exactly like `plan_for` (Step 4). A stamp
mismatch (the registry mutated since, e.g. after `add_providers`) rebuilds. Because a container and every child
share one registry, a resolver is compiled once per registry version for the whole tree.
built on first request by `resolver_compiler.compile_resolver` and then **memoized**, keyed by `provider_id`,
exactly like `plan_for` (Step 4). The memo is cleared whenever the registry mutates (`register` / `add_providers` /
removal), so the next call rebuilds. Because a container and every child share one registry, a resolver is
compiled once for the whole tree and only recompiled after a mutation.

`compile_resolver` dispatches on the provider's concrete type and returns a purpose-built closure:

Expand Down Expand Up @@ -70,18 +70,18 @@ The container provider skips step 2 (it resolves to whichever container is askin
## Step 4 — Wiring plan

The **wiring plan** — the partition of the creator's parameters by how each is satisfied — is consulted when a
`Factory`'s resolver is compiled (once per registry version), not on each resolve. `Factory._plan` delegates to
`Factory`'s resolver is compiled, not on each resolve. `Factory._plan` delegates to
`ProvidersRegistry.plan_for`, which memoizes one `WiringPlan` per
provider on the **registry** — keyed by `provider_id`, stamped with the `ProvidersRegistry.version` it was built
against. A call reuses the plan while that stamp matches the registry's current version and rebuilds when the registry
has changed (i.e. after `add_providers`). Because a container and every child share one `providers_registry`, the plan
is built once per registry version for the whole tree, not once per child container; a `REQUEST`-scoped factory
provider on the **registry** — keyed by `provider_id`. A call reuses the plan until the registry mutates
(i.e. after `register` / `add_providers` / removal), at which point the memo is cleared and the next call rebuilds.
Because a container and every child share one `providers_registry`, the plan
is built once for the whole tree, not once per child container; a `REQUEST`-scoped factory
resolved across many child containers plans only once. `get_dependencies` and `iter_validation_issues` route through the
same memo, so `validate()` warms it rather than rebuilding on each graph walk.

`WiringPlan.build` (in `modern_di/wiring.py`) is a **pure function** of `(parsed_kwargs, kwargs, providers_registry,
owner)` — it reads no cache, scope, or live context, so it runs outside the container lock; the version stamp keeps the
memoized result from going stale against a mutated registry. It is **type matching only**: it decides *which* provider (if any)
owner)` — it reads no cache, scope, or live context, so it runs outside the container lock; clearing the memo on
mutation keeps the memoized result from going stale against a mutated registry. It is **type matching only**: it decides *which* provider (if any)
backs each parameter, not what value that provider currently holds. It iterates `_parsed_kwargs` — the `SignatureItem`
map produced at provider-declaration time by `types_parser.parse_creator` — and sorts each parameter:

Expand Down
15 changes: 7 additions & 8 deletions architecture/validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,12 @@ via a declaration-time `kwargs={...}` is an edge like any type-matched one, and
routed through it is caught here rather than surfacing at resolve time as a bare `RecursionError` or a
`ScopeNotInitializedError`. See [resolution.md](resolution.md) for how the buckets are filled.

**Validated-version short-circuit.** `ProvidersRegistry` carries a `validated_version: int | None`. On a
successful walk `validate()` stamps `validated_version = version`; a later `validate()` whose
`validated_version == version` returns immediately without re-walking, so a repeat `validate()` is free.
Every registry mutation (`register` / `add_providers` / `_remove_providers`) bumps `version` and resets
`validated_version = None`, so any change to the graph re-arms both `validate()` and the runtime guard.
The stamp lives on the registry, which is shared tree-wide, so validating any one container marks the
graph clean for every container in the tree.
**Validated-flag short-circuit.** `ProvidersRegistry` carries a `_validated: bool`, set by `mark_validated()`
on a successful walk; a later `validate()` while `_validated` is still `True` returns immediately without
re-walking, so a repeat `validate()` is free. Every registry mutation (`register` / `add_providers` /
`_remove_providers`) clears `_validated` back to `False`, so any change to the graph re-arms both
`validate()` and the runtime guard. The flag lives on the registry, which is shared tree-wide, so validating
any one container marks the graph clean for every container in the tree.

### Circular dependencies

Expand All @@ -91,7 +90,7 @@ read identically. See [resolution.md](resolution.md#one-renderer) for that drawe
> **Runtime resolution has a cycle guard too — but `validate()` remains the way to see all errors up front.**
> `Container.resolve_provider` wraps the compiled-resolver dispatch (`resolver_for(provider)(self)`) in
> `try/except RecursionError`. The
> handler first short-circuits: if the registry is already validated (`validated_version == version`), the static
> handler first short-circuits: if the registry is already validated (`_validated` is `True`), the static
> graph is known acyclic, so the overflow is genuine self-recursion and the `RecursionError` re-raises untouched
> without any walk. Otherwise, when an unvalidated circular graph's first resolve overflows the stack, the handler
> re-walks the static graph from the failing provider via `DependencyGraph().find_cycle_from` — the same
Expand Down
2 changes: 1 addition & 1 deletion modern_di/container.py
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ def validate(self) -> None:
reg = self.providers_registry
if reg.is_validated():
self._validated = True
return # already validated at this registry version — no re-walk
return # already validated at this registry state — no re-walk

validation_errors: list[Exception] = []
graph = DependencyGraph()
Expand Down
6 changes: 3 additions & 3 deletions modern_di/providers/factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -171,10 +171,10 @@ def _argument_resolution_error(

def _plan(self, container: "Container") -> WiringPlan:
# The plan is memoized on the providers registry (shared by a container and every child), so a
# deeper-scope factory builds it once per registry version, not once per child container. Passing
# deeper-scope factory builds it once and shares it tree-wide, not once per child container. Passing
# the build inputs to plan_for (rather than a closure) keeps the hot cache-hit path allocation-free.
# Building runs outside the container lock — a deterministic function of the registry as of the
# version it was built against, so a race at worst repeats the build (see architecture/concurrency.md).
# Building runs outside the container lock — a deterministic function of the registry's current
# contents, so a race at worst repeats the build (see architecture/concurrency.md).
return container.providers_registry.plan_for(self, self._parsed_kwargs, self._kwargs)

def _resolve_context_value(
Expand Down
84 changes: 39 additions & 45 deletions modern_di/registries/providers_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,39 +14,29 @@


class ProvidersRegistry:
__slots__ = ("_building", "_lock", "_plans", "_providers", "_resolvers", "_version", "validated_version")
__slots__ = ("_building", "_lock", "_plans", "_providers", "_resolvers", "_validated")

def __init__(self) -> None:
self._lock = threading.Lock()
self._providers: dict[type, AbstractProvider[typing.Any]] = {}
self._plans: dict[int, tuple[int, WiringPlan]] = {}
self._resolvers: dict[int, tuple[int, typing.Callable[[Container], typing.Any]]] = {}
self._plans: dict[int, WiringPlan] = {}
self._resolvers: dict[int, typing.Callable[[Container], typing.Any]] = {}
self._building = threading.local() # per-thread compile-in-flight set; the cycle guard is per-call-stack
self._version = 0
self.validated_version: int | None = None
self._validated = False

def __len__(self) -> int:
return len(self._providers)

def __iter__(self) -> typing.Iterator[AbstractProvider[typing.Any]]:
return iter(list(self._providers.values()))

@property
def version(self) -> int:
"""Monotonically increasing counter, bumped on every mutation.

Stamps the memoized `WiringPlan`s held in `_plans` (see `plan_for`) so a plan built
against an older registry is rebuilt instead of resolving against a stale one.
"""
return self._version

def is_validated(self) -> bool:
"""Return whether the graph was validated at the current version — no mutation since the stamp."""
return self.validated_version == self._version
"""Return whether the graph was validated with no registry mutation since."""
return self._validated

def mark_validated(self) -> None:
"""Stamp the current version as validated; a later mutation re-arms it via the version bump."""
self.validated_version = self._version
"""Mark the graph validated; any later mutation clears this."""
self._validated = True

def find_provider(self, dependency_type: type[types.T]) -> AbstractProvider[types.T] | None:
return self._providers.get(dependency_type)
Expand All @@ -57,24 +47,20 @@ def plan_for(
parsed_kwargs: "dict[str, SignatureItem]",
kwargs: dict[str, typing.Any] | None,
) -> "WiringPlan":
"""Return `provider`'s memoized wiring plan, building and stamping it on a miss.

A plan is a pure function of the provider and this registry's contents, so it is
memoized per `provider_id` and stamped with the `version` it was built against; a
stamp mismatch (the registry mutated since) rebuilds. The version is snapshotted
*before* the build runs, so a plan built against a since-mutated registry carries the
old stamp and is never served as current. Shared tree-wide: a container and every
child share one registry, so a deeper-scope provider builds its plan once, not once
per child container. The build inputs are passed by value (not a closure) so the hot
cache-hit path allocates nothing.
"""Return `provider`'s memoized wiring plan, building it on a miss.

A plan is a pure function of the provider and this registry's contents, memoized per
`provider_id` and cleared whenever the registry mutates (`register` / `add_providers` /
removal). Shared tree-wide: a container and every child share one registry, so a
deeper-scope provider builds its plan once, not once per child. Build inputs are passed
by value (not a closure) so the hot cache-hit path allocates nothing.
"""
provider_id = provider.provider_id
version = self._version
cached = self._plans.get(provider_id)
if cached is not None and cached[0] == version:
return cached[1]
if cached is not None:
return cached
plan = WiringPlan.build(parsed_kwargs=parsed_kwargs, kwargs=kwargs, registry=self, owner=provider)
self._plans[provider_id] = (version, plan)
self._plans[provider_id] = plan
return plan

def _building_set(self) -> set[int]:
Expand All @@ -92,15 +78,15 @@ def _building_set(self) -> set[int]:
def resolver_for(self, provider: "AbstractProvider[typing.Any]") -> "typing.Callable[[Container], typing.Any]":
"""Return `provider`'s memoized compiled resolver, building it cycle-safely on a miss.

Memoized and version-stamped exactly like `plan_for`. A back-edge to a provider whose
resolver is still being built (a cycle) captures a thunk that routes through the runtime
`resolve_provider`, so a genuine cycle still raises `RecursionError` -> `CircularDependencyError`.
Memoized per `provider_id` and cleared on registry mutation, exactly like `plan_for`. A
back-edge to a provider whose resolver is still being built (a cycle) captures a thunk that
routes through the runtime `resolve_provider`, so a genuine cycle still raises
`RecursionError` -> `CircularDependencyError`.
"""
pid = provider.provider_id
version = self._version
cached = self._resolvers.get(pid)
if cached is not None and cached[0] == version:
return cached[1]
if cached is not None:
return cached
building = self._building_set()
if pid in building:
return lambda c: c.resolve_provider(provider) # back-edge: route the cycle through runtime
Expand All @@ -109,16 +95,15 @@ def resolver_for(self, provider: "AbstractProvider[typing.Any]") -> "typing.Call
resolver = compile_resolver(provider, self)
finally:
building.discard(pid)
self._resolvers[pid] = (version, resolver)
self._resolvers[pid] = resolver
return resolver

def register(self, provider_type: type, provider: AbstractProvider[typing.Any]) -> None:
with self._lock:
if provider_type in self._providers:
raise exceptions.DuplicateProviderTypeError(provider_type=provider_type)
self._providers[provider_type] = provider
self._version += 1
self.validated_version = None
self._invalidate()

def add_providers(self, *args: AbstractProvider[typing.Any]) -> None:
new_providers: dict[type, AbstractProvider[typing.Any]] = {}
Expand All @@ -134,13 +119,22 @@ def add_providers(self, *args: AbstractProvider[typing.Any]) -> None:
if provider_type in self._providers:
raise exceptions.DuplicateProviderTypeError(provider_type=provider_type)
self._providers.update(new_providers)
self._version += 1
self.validated_version = None
self._invalidate()

def _remove_providers(self, *provider_types: type) -> None:
"""Rollback helper for `Container.add_providers`; not part of the public API."""
with self._lock:
for provider_type in provider_types:
self._providers.pop(provider_type, None)
self._version += 1
self.validated_version = None
self._invalidate()

def _invalidate(self) -> None:
"""Drop the memoized plans/resolvers and the validation flag — the registry changed.

Called under `self._lock` by every mutation. Clearing has the same breadth the old version
bump did (a bump invalidated every memo anyway) and frees stale entries eagerly. Sound
because mutation is a single-threaded configure-phase operation (architecture/concurrency.md).
"""
self._plans.clear()
self._resolvers.clear()
self._validated = False
Loading
Loading