elastic / elastic/beats

[bug-hunter] Kafka output accepts headers for unsupported version 0.9

Open Beginner friendly
#49,538 2 comments 0 reactions 0 assignees View on GitHub
needs_team
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

## Impact
Users can configure Kafka output with `version: "0.9"` and non-empty `headers`, and Beats accepts the config instead of failing validation. Headers are only supported for Kafka >= 0.11, so this allows an invalid configuration that can fail at runtime against older brokers.

## Reproduction Steps
1. Save this script as `/tmp/gh-aw/agent/repro_kafka_headers.go`:
```go
package main

import (
"fmt"
"os"

kafkaout "github.com/elastic/beats/v7/libbeat/outputs/kafka"
"github.com/elastic/elastic-agent-libs/config"
"github.com/elastic/elastic-agent-libs/mapstr"
)

func main() {
c := config.MustNewConfigFrom(mapstr.M{
"hosts": []string{"localhost:9092"},
"topic": "test-topic",
"version": "0.9",
"headers": []mapstr.M\{\{"key": "k", "value": "v"}},
})

_, err := kafkaout.ReadConfig(c)
if err != nil {
fmt.Printf("expected error path triggered: %v\n", err)
os.Exit(0)
}

fmt.Println("BUG: ReadConfig accepted headers with kafka version 0.9")
os.Exit(1)
}
```
2. Run:
```bash
cd /home/runner/work/beats/beats
go run /tmp/gh-aw/agent/repro_kafka_headers.go
```

## Expected vs Actual
**Expected:** `ReadConfig` rejects the config with an error because headers are unsupported for Kafka versions `< 0.11`.

**Actual:** Config is accepted and script prints:
```text
BUG: ReadConfig accepted headers with kafka version 0.9
exit status 1
```

## Failing Test
The reproduction script above is a minimal failing test case.

## Evidence
- Validation compares versions as strings in `libbeat/outputs/kafka/config.go#L188-L190`:
- `if len(c.Headers) != 0 && c.Version < kafka.Version("0.11") { ... }`
- `kafka.Version` is `type Version string` in `libbeat/common/kafka/version.go#L27`, so `<` is lexical string comparison.
- Lexical compare misorders `"0.9"` vs `"0.11"` (`"0.9" > "0.11"` lexically), so invalid config bypasses the guard.
- Duplicate check: searched issues/PRs for this exact validation message and scenario; no matching open report found.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/beats/actions/runs/23242113188)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Mar 25, 2026, 11:37 AM UTC

Contributor guide

Open the contributing guide

Research direction

Start in libbeat/outputs/kafka/config.go around the headers validation at lines 188-190, then read libbeat/common/kafka/version.go to confirm how Version values are represented. Run the reproduction at /tmp/gh-aw/agent/repro_kafka_headers.go and add coverage for version 0.9 with non-empty headers. Done means ReadConfig rejects this configuration while supported versions remain valid.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
backend, stream-processing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.