apache / apache/pinot

Reset the consumer offsets for a table actively ingesting

Open
#6,236 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Usecase: Realtime table where the consumer starts lagging behind (say 3-4 hours). User wished to reset the consumer offsets to latest. They were okay with losing data of the last 3-4 hours.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by tracing Pinot’s realtime table consumer-offset handling and clarify how a reset-to-latest operation should behave while ingestion continues. Done means the behavior is defined and the reported data-loss case is covered by appropriate tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.