dell / dell/iDRAC-Telemetry-Reference-Tools
Separate iDRAC telemetry into distinct Kafka topics: logs and metrics
- Dominant language
- Go
- Stars
- 44
- Forks
- 32
- PR merge metrics
- No merged PRs in 30d
Description
## Problem Statement
Omnia currently publishes all iDRAC telemetry to a single Kafka topic (`idrac`). This mixes structured metrics and unstructured logs in the same stream, which complicates downstream processing:
- Consumers must parse and filter mixed payloads to extract only metrics or only logs
- Metrics pipelines ingest log messages (noise) and log pipelines ingest metric messages (noise)
- No clean separation for retention, partitioning, or access control at the data type level
## Current Behavior
- One Kafka topic: `idrac`
- iDRAC telemetry producer pushes both metrics and logs to the same topic
- Consumers receive a mixed stream and must filter by message type
## Desired Behavior
- Two Kafka topics for iDRAC:
- `idrac-metrics` for structured metric data
- `idrac-logs` for log/trace/event data
- Producer routes messages by type to the appropriate topic
- Consumers can subscribe to the relevant topic only
Contributor guide
Research direction
Start by locating the iDRAC telemetry producer and consumers that use the current `idrac` Kafka topic, then trace how messages are identified as metrics or logs. Done means the producer routes structured metrics to `idrac-metrics` and log/trace/event data to `idrac-logs`, allowing consumers to subscribe without filtering mixed messages.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100