apache / apache/flink-agents

[Tech Debt][runtime] Replace isDone() polling with completion-callback wakeup for suspended Java async actions

Open
#1,109 2 comments 0 reactions 1 assignee Claimed by @ysymi View on GitHub
priority/major tech debt
Dominant language
Java
Stars
452
Forks
167
Avg merge
5d 9h
Merged PRs (30d)
49

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar.

### Description

## Background

When a Java action suspends on `executeAsync` (e.g. waiting for an LLM call), `ActionExecutionOperator` resumes it by **polling**: it unconditionally re-submits a
`tryProcessActionTaskForKey` mail, which checks `pending.isDone()` and re-queues if not done.

## Problem

There is no backoff between poll rounds, so the cost depends entirely on mailbox traffic:

- **Busy mailbox** (many concurrent keys): poll mails interleave with real work — waste is amortized, barely noticeable.
- **Idle mailbox** (few keys, long async waits — typical for LLM inference workloads): the re-submitted mail is processed almost immediately, degrading to near busy-waiting for the entire
suspension (seconds).

Each poll round pays a full RocksDB list read + write-back in `pollNextActionTask` / `removeProcessingKey`, even when the future is not done and no progress is possible. For workloads
where every record suspends on an LLM call, this is a per-record, per-poll-round I/O tax.

## Proposal

Let the completion do the wakeup instead of the mailbox checking for it:

- Register a one-shot `whenComplete` callback on the async future; when it completes, the worker thread submits **exactly one** wakeup mail keyed by the suspended record's Flink key.
- The operator's own re-submit step then only runs when there is no callback-owned wakeup pending for the key — the wakeup has a single owner at any time. (This single-owner rule is the
correctness-critical part: both paths firing for one suspension leads to a duplicate resume mail, which surfaces as the `removedCount == 1` checkState in `processActionTaskForKey` failing
on cleanup.)
- Polling remains as fallback when no callback is registered (tests, local environments).

## Notes

- Unit tests don't cover this path: surefire's `target/classes` classpath doesn't resolve `META-INF/versions/21`, so the JDK21 continuation variant is never loaded — needs e2e/driver-level
verification.
- We have a working, validated implementation and are happy to contribute it as a PR.

We'd like feedback on whether the callback-wakeup model is something the community wants to adopt.

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.