elastic / elastic/beats

[Filebeat] Add health status reporting to Kafka input

Open
#44,650 2 comments 0 reactions 0 assignees View on GitHub
enhancement Filebeat Stalled Team:Elastic-Agent-Data-Plane
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

Update the Kafka input to report its health status, providing better visibility into the operational state for users.

The Kafka input should report its health status using the [`context.UpdateStatus()`](https://github.com/elastic/beats/blob/9190f776f5fb612a880cb013a9eec8a49fe17afd/filebeat/input/v2/input.go#L112) method. Status changes should only be reported when the status actually changes (don't repeatedly send HEALTHY if the status is already HEALTHY).

### Status Reporting
The input should report the following states:

- **STARTING**: TBD
- **CONFIGURING**: TBD
- **HEALTHY**: TBD
- **DEGRADED**: TBD
- **FAILED**: TBD

Include relevant context information with non-HEALTHY states:

- TBD

## References
See the CEL input implementation for reference:
https://github.com/elastic/beats/blob/6409d005a31e0147a40f0872794880b3df15a69c/x-pack/filebeat/input/cel/input.go#L111

Contributor guide

Open the contributing guide

Research direction

Start with the Kafka input implementation and the context.UpdateStatus method in filebeat/input/v2/input.go, then compare the referenced CEL implementation in x-pack/filebeat/input/cel/input.go. The status meanings and non-HEALTHY context are still TBD; done requires defining those states and reporting changes only when the status differs.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
observability, stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.