temporalio / temporalio/temporal

Avoid writing history for starts that will be rejected/deduplicated by the workflow-id reuse/conflict policy

Open
#12,103 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
23.2k
Forks
1.9k
Avg merge
2d 8h
Merged PRs (30d)
228

Description

When StartWorkflowExecution targets an existing workflow id, the server writes the new run's first history batch (createBrandNew → AppendHistoryNodes) before it resolves the workflow-id reuse/conflict policy. If the policy then rejects or deduplicates the start — REJECT_DUPLICATE, ALLOW_DUPLICATE_FAILED_ONLY on a successfully-completed prior, conflict FAIL, USE_EXISTING, or the terminate paths — that history is already in db and is orphaned.
Reclaiming this orphaned history is then job of history scanner only.

This has two costs for self-hosted clusters:

  • Write pressure on persistence — every rejected start still performs a history write.
  • Storage growth until the scavenger catches up (which runs at most every 12h and skips history younger than a 60-day default)

It's most visible on workloads that use the workflow id for deduplication (e.g. REJECT_DUPLICATE at volume) or that repeatedly start an id while a run is active (conflict FAIL).

1.32.0 added history.businessIDReuseRate, which is checked before the history write — but it's a rate limit, so it only helps the rapid-repeat / hot-id case. It does not apply to policy-based rejections/dedups, which still write history first.

Requested change:

Extend pre-write gating (in the spirit of businessIDReuseRate) to the reuse/conflict-policy paths: when a start targets an existing id, resolve whether it will be rejected/deduplicated before appending history, so doomed starts don't write (and orphan) history.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the StartWorkflowExecution path and trace createBrandNew → AppendHistoryNodes, then inspect where workflow-id reuse and conflict policies are resolved. Verify each rejected or deduplicated path, including terminate paths, before changing the ordering. Done means doomed starts no longer append history while accepted starts retain their initial history.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend-api-design, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.