experiment separate Broker + Router#624
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #624 +/- ##
==========================================
+ Coverage 80.20% 88.69% +8.48%
==========================================
Files 69 82 +13
Lines 2577 3814 +1237
==========================================
+ Hits 2067 3383 +1316
+ Misses 510 431 -79 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
s3rius
left a comment
s3rius
left a comment
There was a problem hiding this comment.
Well. Although the intention is good for sure.
But I see several problems with the solution you provided. Most importantly I see that brokers are referenced by strings almost everywhere. Which is problematic.
Also, since everything is implicit, it's hard to resolve conflicts in this setup. Like, what if I have multiple routers. I guess by design it's highly advised to only use one instance of router per project, but most probably people would not care about this constraint, since it's not enforced.
I'm not quite sure about Flow concept. Although multiple queue support is highly anticipated by the community, and I'm fully up for its support. But this implementation lacks some fundamental abilities for future expansion. Different brokers might have their own different parameters on Queues. For example RabbitMQ has lots of things that it can do with a queue.
And I'm not quite sure about flow_kind. It seems a bit redundant. Or I didn't catch the meaning of it.
I still like the intention. But I guess it need to be redesigned.
GefMar
force-pushed
the
experiment/separate_broker
branch
from
95714b0 to
8805f3d
Compare
Introduce router-owned task routing, flow subscriptions, and shared task declarations while preserving the existing broker-first task API. Add transport-neutral Flow contracts, explicit route/subscription semantics, multi-broker router ownership, task_builder base_cls support, prepared invocation route snapshots, and scheduler/requeue compatibility behavior. Document migration guidance and add executable routing examples for multiple brokers and shared task packages.
Introduce Flow, TaskiqRouter, route and subscription contracts to move task dispatch decisions out of brokers while keeping brokers as transport adapters. Preserve backward-compatible
Remove direct router access to broker private task storage and add an explicit broker-owned storage boundary for router-managed task registration. Clarify prepared route snapshots, subscription task-name semantics and setup-time router mutation contracts. Extend routing docs with flow resolution precedence and custom task base class middleware behavior. Add regression coverage for shared task local registration, custom base_cls middleware hooks and scheduler fallback when a route is removed before dispatch.
GefMar
force-pushed
the
experiment/separate_broker
branch
from
426187e to
cc6d3a3
Compare
- separate routing policy, task registration, flows, and broker transport - support explicit multi-broker workers with shared process limits - preserve legacy decorators, scheduling, requeue, and send semantics - harden shutdown, cancellation, delivery ownership, and shared task binding - add compatibility tests, examples, and migration documentation
- manage shared router brokers through the scheduler lifecycle - reject unsupported explicit flows instead of silently discarding them - preserve legacy no-flow and in-memory dispatch behavior - add routing, lifecycle, compatibility tests, and documentation
- keep immediate retries on the receiving broker - protect callback cleanup from repeated cancellation - make scheduler rollback cleanup idempotent - document unsupported-flow requeue behavior
- cover receiver and worker runtime cancellation and shutdown paths - validate scheduler, router, registry, CLI, and broker lifecycle contracts - support callable-object shared tasks during broker binding - organize runtime fault tests into focused modules
2 participants
No description provided.