apache / apache/fluss

[server] Bound per-TS sender queues and handle stale retries

Open
#4,291 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Description

Follow-up to #3391 and #4262, to be completed before migrating production
control RPCs to the sender framework.

The current queue is unbounded, and transport failures retain the head
indefinitely while the TS remains registered. This can accumulate pending
requests. Epoch validation also occurs only at dequeue, not during retries.

### Scope

- Add configurable queue capacity and non-blocking admission with an explicit
rejection policy, so saturation cannot block the Coordinator event thread.
- Check request validity throughout retry and connection-cleanup paths,
including epoch changes, leadership loss, and channel removal.
- Preserve FIFO for valid requests with unknown outcomes. Define safe retirement
of stale requests rather than silently dropping requests after a retry limit.

### Validation

Cover queue saturation, non-blocking admission, stale-request retirement,
leadership/channel teardown, RPC cleanup, and FIFO preservation.

Concrete RPC migration and business recovery logic remain separate follow-ups.

Review: https://github.com/apache/fluss/pull/3391#issuecomment-5613908184

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with follow-ups #3391 and #4262 and trace the sender framework's queue, retry, connection-cleanup, and channel-removal paths; the issue names no source files or entry points. Use focused tests for saturation, non-blocking admission, stale-request retirement, teardown cleanup, and FIFO behavior to define done.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.