lablup / lablup/backend.ai

Refactor the event dispatcher: explicit event registry and grouped handler registration

Open
#13,812 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

### Goal

Make the event dispatcher's knowledge of the event catalogue and of its handlers explicit, so that delivery guarantees hold structurally rather than by convention.

### Background

Follow-up work split out of the Pydantic event payload epic (BA-7312). That epic's own goal - removing msgpack from the event path - is complete; what remains are dispatcher concerns raised during its review, which are about registration and dispatch rather than serialization.

### Problems

- No single place knows every event type. Broadcast events self-register into AbstractBroadcastEvent._register_dict as an import-time side effect of __init_subclass__; anycast events are not registered at all. Nothing can enumerate the catalogue.
- Callback signatures carry source and context. context is None at 76 of 86 registrations and handlers ignore it; source is used as a domain identifier and belongs in the event body instead.
- Handler registration is a flat list of about 75 consume() / subscribe() calls in the manager dispatch module, with no grouping by domain.
- The consume() docstring promises that only one callback runs per event, but the implementation dispatches every registered callback and waits for all of them before acking, so an anycast event registered twice is processed twice.

### Scope

- An explicit event-type registry on the dispatcher covering both anycast and broadcast, replacing _register_dict and the __init_subclass__ side effect.
- Removal of source and context from the callback signature, with producing-node identity moved into the event body.
- Per-domain handler groups shaped like the service processor groups, built on the existing EventDispatcherGroup.
- Enforcement of one handler per event at registration time.

### Order

The registry lands first: the grouped-registration and uniqueness work depends on a known set of event types.

JIRA Issue: BA-7390

Contributor guide

Open the contributing guide

Research direction

Start by tracing AbstractBroadcastEvent._register_dict and __init_subclass__, then inspect the manager dispatch module, EventDispatcherGroup, and existing consume()/subscribe() registrations. Done means both event kinds are covered by an explicit registry, callbacks no longer take source or context, handlers are grouped by domain, and duplicate handlers are rejected at registration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, backend-api-design, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.