fluent / fluent/fluent-bit

Router: conditional routes sharing an output — silent drop vs. duplicate delivery

Open
#12,144 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
8.1k
Forks
2k
Avg merge
4d 20h
Merged PRs (30d)
71

Description

## Bug Report / Design question

**Describe the bug**

In the input-level conditional router, when **two or more routes target the same output instance**, Fluent Bit cannot deliver correctly, and the current data model appears to make a fully-correct outcome impossible without a redesign. I'd like maintainer input on the intended semantics and on a proposed fix before sending a large change.

Example config intent (one input, two named conditional routes, **same destination** `es`):

```yaml
pipeline:
inputs:
- name: dummy
routes:
logs:
- name: error_logs
condition: { rules: [ { field: "$level", op: eq, value: "error" } ] }
to: [ es ]
- name: checkout_logs
condition: { rules: [ { field: "$service", op: eq, value: "checkout" } ] }
to: [ es ]
outputs:
- name: es
```

**Current behavior**

`flb_router_apply_config()` deduplicates direct routes by output instance only (`input_has_direct_route()` in `src/flb_router_config.c`). Because `routes_direct` ends up with a single path for `es`, `split_and_append_route_payloads()` (`src/flb_input_log.c`) builds a per-route payload only for the first route — **the second route is silently dropped**. A record matching `checkout_logs` but not `error_logs` never reaches `es`.

**Why the naive fix does not work**

Making the dedupe key `(route, output)` instead of `output` stops the silent drop, but the conditional-routing data model assumes **at most one route per output**:

1. `split_and_append_route_payloads()` builds **one chunk per route** (`payload->route`, tag = route name) and `build_payload_for_route()` re-evaluates each route's condition independently with **no cross-route dedup**. A record matching both routes is encoded into both payload chunks, so the shared output `es` receives it **twice** (two separate chunks/tasks — this cannot be deduped at the task layer).
2. `routes_mask` and `fs_chunks_size` accounting are keyed by **output id**. With two routes on one output, `route_payload_apply_outputs()` subtracts the shared output's `fs_chunks_size` for the "other" route even though the payload does route there (and never adds it back while `fs_counted` stays true), **undercounting storage** and bypassing `total_limit_size` pressure.

So the choice today is between **under-delivery** (silent drop, current) and **over-delivery + broken storage accounting** (naive fix). The only correct semantic is: *each output receives each record at most once, if any route targeting it matches (A OR B).* A per-route payload split with a uniform per-chunk `routes_mask` cannot express that when routes have multiple, overlapping output sets.

**Expected behavior**

Each output should receive each record exactly once when one or more routes targeting it match; no route should be silently dropped.

**Proposed design (seeking feedback)**

Group records by their computed **output set** instead of by route:

- For each record, compute `S = union of outputs(R)` over all matching routes `R` (falling back to default-route outputs when no non-default route matches).
- Emit **one chunk per distinct non-empty `S`**, with `routes_mask = S`.

This preserves the output-keyed mask/storage invariant (so the accounting bug disappears), drops no route, and duplicates no delivery.

**Open question — chunk tag semantics**

Today the delivered chunk tag is the **route name**, which is user-visible (out_file naming, ES index, downstream `Match`). In the merged model a record in `S = {es, s3}` has no single route, so the tag must be redefined. Candidates:

- **T1**: original input tag (simplest; loses route-name tagging even for single-route records).
- **T2**: deterministic "primary" route = first matching route in config order (identical to today whenever a record matches exactly one route; documented tie-break for overlaps).
- **T3**: synthetic tag from route/output names.

T2 seems least disruptive, but this is a semantics change we don't want to decide unilaterally.

**Questions for maintainers**

1. Is "multiple routes → same output" a configuration you intend to support, or should it be explicitly rejected/warned at config load?
2. If supported, is the per-output-set grouping the direction you'd want, and which tag semantics (T1/T2/T3) is acceptable?
3. Is there existing intent/design for this case we should align with?

**Additional context**

- Related PR (contains the analysis and an interim `(route, output)` dedupe that surfaced the P1/P2 issues above): fluent/fluent-bit#12142
- Relevant code: `src/flb_router_config.c` (`flb_router_apply_config`, `input_has_direct_route`), `src/flb_input_log.c` (`split_and_append_route_payloads`, `build_payload_for_route`, `route_payload_apply_outputs`), `src/flb_task.c` (`flb_task_create`).

**Your Environment**
* Version used: master (commit cb7256c28 base)

Contributor guide

Open the contributing guide

Research direction

Read the related PR #12142, then trace flb_router_apply_config() and input_has_direct_route() in src/flb_router_config.c alongside split_and_append_route_payloads(), build_payload_for_route(), and route_payload_apply_outputs() in src/flb_input_log.c. Compare the output-keyed routing and storage accounting with flb_task_create() in src/flb_task.c; done means an agreed semantic for shared outputs and chunk tags, or an explicit configuration rejection/warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
backend, stream-processing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.