fluent / fluent/fluent-bit

Support the $TAG record accessor in processor/filter condition rules

Open
#12,103 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
8.1k
Forks
2k
Avg merge
4d 16h
Merged PRs (30d)
58

Description

**Is your feature request related to a problem? Please describe.**

Processor and filter condition rules cannot match on the record's **tag**.

The record accessor already understands `$TAG` / `$TAG[n]`, and conditions
accept record-accessor patterns for record/metadata fields. But the condition
evaluator calls `flb_cfl_ra_translate()` with a `NULL` tag
(`src/flb_conditionals.c`, `evaluate_rule()`), so any rule referencing `$TAG`
resolves to nothing and silently never matches. There is no error — the rule
just always evaluates false, which is surprising given `$TAG` works elsewhere.

Concretely, this does not work today:

```yaml
processors:
logs:
- name: content_modifier
action: insert
key: tier
value: frontend
condition:
op: and
rules:
- field: "$TAG"
op: regex
value: "^app\.frontend\..*$"
```

**Describe the solution you'd like**

Allow condition rules to reference `$TAG` by making the chunk tag available to
the condition evaluator, so a processor/filter can select records by tag inline
(e.g. to consolidate several tag-routed streams into one output without an
extra routing hop).

I put together a reference implementation in #12090 (draft) that threads the
chunk tag through the evaluation path (`flb_condition_evaluate[_ex]`,
`flb_mp_chunk_cobj`, the processor and filter callers; the router keeps passing
`NULL` since it routes by tag natively) and adds unit coverage. **I opened it
mainly to make the design concrete for discussion, not as a finished proposal** —
happy to change the approach.

Questions for maintainers:

1. Is enabling `$TAG` inside processor/filter conditions a direction you'd
accept, or is matching-by-tag intended to stay solely at the routing layer
(`Match`, routing conditions)?
2. If yes, is threading `tag`/`tag_len` through `flb_condition_evaluate*` the
preferred shape, or would you rather the tag be carried some other way
(e.g. on an existing context struct) to avoid the signature change?

**Describe alternatives you've considered**

- `rewrite_tag` + separate pipelines / `Match` selection — works, but forces
pipeline fan-out for what is conceptually a single conditional processor step.
- Routing-level conditions — select at the router, not within a processor, so
they can't gate an in-place processor action on the tag.

**Additional context**

Use case: several inputs land under different tags and need a single processor
action applied only to a subset selected by tag before sharing one output.
Today that requires restructuring the pipeline; a `$TAG` condition expresses it
directly.

Contributor guide

Open the contributing guide

Research direction

Start with src/flb_conditionals.c and evaluate_rule(), then review the evaluation entry points and processor/filter callers named in the issue alongside the reference implementation in #12090. Confirm the preferred way to make the chunk tag available and add coverage showing that $TAG conditions match processor and filter records without changing router behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.