temporalio / temporalio/temporal

Reconsider the 60-day default for worker.historyScannerDataMinAge (or document its rationale)

Open
#12,102 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
23.2k
Forks
1.9k
Avg merge
2d 8h
Merged PRs (30d)
228

Description

worker.historyScannerDataMinAge (default 60 days) sets how old an orphaned history branch must be before the history scanner ("scavenger") will reclaim it. Orphaned branches include history left behind by rejected or deduplicated StartWorkflowExecution calls.

With the 60-day default, this leftover — which is pure garbage once the workflow record is gone — sits for up to two months before the scavenger will touch it. On clusters where it accumulates (for example high-volume workflow-id-based deduplication with REJECT_DUPLICATE), operators typically need to lower this to hours to keep the history_node / history_tree tables from growing.

Request here is to lower default of worker.historyScannerDataMinAge to something less conservative. One week would imho be OK option.
Reason is that currently most of on-prem users don't know this dynamic exists so they do not explicitly change it and just use default.
There is no real guidance on how low this dynamic config should also be set. Having some guidance would really help as well.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the definition and uses of worker.historyScannerDataMinAge, then read how the history scanner reclaims orphaned branches. Determine an appropriately lower default and add guidance explaining the setting and its tradeoffs. Done means the default and operator documentation reflect the decision, with relevant validation for history_node and history_tree cleanup.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, databases, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.