dell / dell/iDRAC-Telemetry-Reference-Tools

Separate iDRAC telemetry into distinct Kafka topics: logs and metrics

Open
#182 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
44
Forks
32
PR merge metrics
No merged PRs in 30d

Description

## Problem Statement
Omnia currently publishes all iDRAC telemetry to a single Kafka topic (`idrac`). This mixes structured metrics and unstructured logs in the same stream, which complicates downstream processing:
- Consumers must parse and filter mixed payloads to extract only metrics or only logs
- Metrics pipelines ingest log messages (noise) and log pipelines ingest metric messages (noise)
- No clean separation for retention, partitioning, or access control at the data type level

## Current Behavior
- One Kafka topic: `idrac`
- iDRAC telemetry producer pushes both metrics and logs to the same topic
- Consumers receive a mixed stream and must filter by message type

## Desired Behavior
- Two Kafka topics for iDRAC:
- `idrac-metrics` for structured metric data
- `idrac-logs` for log/trace/event data
- Producer routes messages by type to the appropriate topic
- Consumers can subscribe to the relevant topic only

Contributor guide

Open the contributing guide

Research direction

Start by locating the iDRAC telemetry producer and consumers that use the current `idrac` Kafka topic, then trace how messages are identified as metrics or logs. Done means the producer routes structured metrics to `idrac-metrics` and log/trace/event data to `idrac-logs`, allowing consumers to subscribe without filtering mixed messages.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.