temporalio / temporalio/temporal
Reconsider the 60-day default for worker.historyScannerDataMinAge (or document its rationale)
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 23.2k
- Forks
- 1.9k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 228
Description
worker.historyScannerDataMinAge (default 60 days) sets how old an orphaned history branch must be before the history scanner ("scavenger") will reclaim it. Orphaned branches include history left behind by rejected or deduplicated StartWorkflowExecution calls.
With the 60-day default, this leftover — which is pure garbage once the workflow record is gone — sits for up to two months before the scavenger will touch it. On clusters where it accumulates (for example high-volume workflow-id-based deduplication with REJECT_DUPLICATE), operators typically need to lower this to hours to keep the history_node / history_tree tables from growing.
Request here is to lower default of worker.historyScannerDataMinAge to something less conservative. One week would imho be OK option.
Reason is that currently most of on-prem users don't know this dynamic exists so they do not explicitly change it and just use default.
There is no real guidance on how low this dynamic config should also be set. Having some guidance would really help as well.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the definition and uses of worker.historyScannerDataMinAge, then read how the history scanner reclaims orphaned branches. Determine an appropriately lower default and add guidance explaining the setting and its tradeoffs. Done means the default and operator documentation reflect the decision, with relevant validation for history_node and history_tree cleanup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100