BrighterCommand / BrighterCommand/Brighter

Support multiple outbox providers with different transaction types

Open
#4,133 1 comment 0 reactions 0 assignees View on GitHub
0 - Backlog Breaking Change feature request under_discussion V11
Dominant language
C#
Stars
2.5k
Forks
296
Avg merge
1d 11h
Merged PRs (30d)
21

Description

## Problem

Brighter currently assumes a single outbox provider per application host. The `ProducersConfiguration` binds one `IAmAnOutbox` and one `TransactionProvider` (type) globally. When a user needs to use multiple outbox providers with different transaction types (e.g. Postgres with `DbTransaction` + MongoDB with its own transaction type), the current architecture cannot support this — the DI registrations overwrite each other.

This was reported in #3878 (ergonomic issue with `UseOutboxArchiver` requiring compile-time knowledge of the transaction type) and surfaced again in #4114 (attempted fix that highlighted the deeper limitation).

## Current Architecture

The **domain classes already support per-instance outboxes**:

- `CommandProcessor` holds its own `IAmAnOutboxProducerMediator` as an instance field (`CommandProcessor.cs:132`), not a static/shared field
- `OutboxProducerMediator` holds its own `IAmAnOutboxSync` / `IAmAnOutboxAsync` references as instance fields (`OutboxProducerMediator.cs:57-58`)

The constraint is **entirely in the DI registration and configuration layer**:

1. **`IAmAnOutbox`** — registered as a singleton via `ServiceCollectionExtensions.AddProducers`. Each call uses `.Add()` so the last registration wins
2. **`IAmAnOutboxProducerMediator`** — registered as a singleton via `TryAdd`, so the first registration wins and subsequent calls are ignored
3. **`IAmABoxTransactionProvider`** — singleton registration, one type per app
4. **`ProducersConfiguration` / `IAmProducersConfiguration`** — has a single `Outbox` property and single `TransactionProvider` property, binding one outbox globally

## What Multi-Outbox Would Require

### 1. Per-Publication (or Per-Producer) Outbox Binding

The `ProducersConfiguration` needs to support associating different outbox/transaction-provider pairs with different producers or publications. Today these are global properties on `IAmProducersConfiguration`:

```csharp
IAmAnOutbox? Outbox { get; set; }
Type? TransactionProvider { get; set; }
```

This could evolve to allow per-producer configuration, where each producer entry in the `ProducerRegistry` can optionally specify its own outbox and transaction provider. A global default could remain for backward compatibility, with per-producer overrides.

### 2. DI Registration Changes

`ServiceCollectionExtensions.AddProducers` currently registers a single `IAmAnOutboxProducerMediator` singleton. For multi-outbox, this needs to either:

- Register **multiple mediator instances** (e.g. using .NET 8 keyed services, keyed by transaction type or producer name)
- Or register a **composite mediator** that internally delegates to the correct outbox based on the publication/producer being used

The `CommandProcessor` already holds its mediator as an instance field, so the wiring change is about resolving the right mediator — not restructuring the processor itself.

### 3. Sweeper and Archiver

`TimedOutboxSweeper` resolves a single `IAmAnOutboxProducerMediator` from DI. `TimedOutboxArchiver` is bound to a single transaction type.

Options:
- **Multiple hosted services** — register one sweeper/archiver per outbox provider. This is the simplest model and aligns with the singleton-per-type pattern
- **Composite pattern** — a single sweeper that iterates over all registered outboxes
- **Document the separate-process model** — for users who prefer deployment isolation, recommend running separate sweeper instances per outbox (this already works today and may be the pragmatic recommendation)

## Breaking Changes

This requires changes to **public interfaces**, making it a V11 change:

- `IAmProducersConfiguration` — the `Outbox` and `TransactionProvider` properties would need to evolve (either made optional with per-producer alternatives, or the interface extended)
- `AddProducers` API surface — the extension method signatures and registration behavior would change
- `UseOutboxArchiver` / `UseOutboxSweeper` — may need overloads or changes to support multiple outbox types

## Related Issues / PRs

- #3878 — Original report: `UseOutboxArchiver` should not require the generic type parameter
- #4114 — PR adding non-generic `UseOutboxArchiver()` overload (addresses ergonomics but not the underlying multi-outbox limitation)

## Design Considerations

- Should the outbox be bound at the **publication** level (per-topic) or the **producer** level? Publication seems more natural since it's the unit of "what message goes where"
- .NET 8 keyed services could simplify the DI resolution pattern
- Backward compatibility: a single global outbox configuration should remain the simple/default path; multi-outbox is an advanced scenario
- The existing `InMemoryOutbox` default when no outbox is configured should continue to work unchanged

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.