ISISComputingGroup / ISISComputingGroup/DataStreaming

`kafka_event_aggregator`: backpressure

Open
#111 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

If producer queue fills beyond N bytes, pause consumer until it recovers to less than M bytes (and provide warnings + metrics about consumer pause).

(need to consider behaviour on pausing consumer mid-frame - we are likely to get partial frames in that case)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the kafka_event_aggregator entry point and trace how producer queue size, consumer reads, and frame boundaries are handled. Define the N/M thresholds and confirm how pausing mid-frame should work; done means consumer pause and recovery, warnings, and pause metrics are implemented without losing or mishandling partial frames.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka
Domain
stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.