temporalio / temporalio/temporal
Task start can remain undelivered after an ambiguous History timeout
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 23.2k
- Forks
- 1.9k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 228
Description
Expected Behavior
If History commits a workflow- or activity-task start while the original worker poll is still active, Matching should be able to recover the committed response through History's existing same-request-ID contract. A per-attempt RPC deadline should not leave the task started in History but undelivered to the worker.
Actual Behavior
Matching uses a shorter child context for RecordWorkflowTaskStarted and RecordActivityTaskStarted than the parent worker long poll. History can commit the start while Matching observes the child-context deadline before receiving the response.
History already treats a repeated request ID as idempotent. However, once Matching leaves that concrete task-to-poller dispatch, a later selection creates a new request ID. History then returns TaskAlreadyStarted, Matching drops that delivery, and no worker receives the committed start response. Recovery is delayed until the workflow-task or activity timeout.
Steps to Reproduce the Problem
- Match a workflow or activity task to a worker poll.
- Allow
Record*TaskStartedto commit in History, but delay its response until Matching's child context expires. - Keep the original worker poll active and allow the task-start path to recover.
- Observe that leaving the original dispatch causes a later delivery to use a different request ID, receive
TaskAlreadyStarted, and fail to deliver the committed start response.
Specifications
- Version: current
main - Platform: server-side behavior; platform independent
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the Matching task-start path for RecordWorkflowTaskStarted and RecordActivityTaskStarted, focusing on the child RPC deadline and request-ID handling. Reproduce the delayed History response described in the issue, then verify that a committed task start is delivered to the still-active worker poll rather than retried with a new request ID and dropped as TaskAlreadyStarted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100