temporalio / temporalio/temporal
Gradual change subscriber misses the transition when its timer fires at the When() time
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 23.2k
- Forks
- 1.9k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 228
Description
What are you really trying to do?
Rely on GradualChange-based dynamic config rollouts (e.g. the new-matcher and fairness rollouts consumed by task_queue_partition_manager) to actually deliver the New value to subscribers at their per-key switch time.
Describe the bug
gradual_change.go computes the Old/New decision and the switch time with different rounding:
Valuecomputes a truncateduint32threshold from the current time and uses strictfingerprint < threshold.Whenmaps the fingerprint back to a time.
As a result Value(key, When(key)) is deterministically still Old, and remains Old for up to (End-Start)/2^32 after When (~200µs for a 10-day rollout, ~7ms for a 1-year one).
The subscription wrapper (reevalLocked) only schedules a new timer when When(key) is strictly in the future. When the runtime timer fires within that quantum of its deadline — which is common, Go timers routinely fire within sub-millisecond of the deadline — the wrapper reevaluates, still sees Old, schedules nothing, and the subscriber never receives the New value until some unrelated config change triggers a reevaluation.
Minimal Reproduction
Two unit tests demonstrate this on main (706e0b4):
Value(key, When(key))returns"old"for every key tested (expected"new").- A subscription test with a fake clock advanced exactly to the timer deadline never observes the transition (
Condition never satisfied).
Environment/Versions
- Temporal Version: main (
706e0b4), darwin/arm64 (not platform-specific)
I will submit a PR that derives the per-key switch time once and uses it from both Value and When, making Value(key, When(key)) == New by construction.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with gradual_change.go and the two minimal-reproduction unit tests described in the issue. Trace how Value and When calculate the per-key transition, then verify that Value(key, When(key)) returns the New value and that a subscription observes the transition when the timer reaches its deadline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100