tektoncd / tektoncd/pipelines-as-code
periodically resync the in-memory concurrency queue with the cluster
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 214
- Forks
- 144
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 27
Description
📝 Description
Every concurrency bug fixed in #2890, and every one found in its post-merge
review, has the same shape: the queue the watcher holds in memory drifts away
from what is actually running in the cluster. Each fix closes one path to
drift. None of them heals drift that has already happened, or drift from a
path nobody has found yet. Today the only recovery is restarting the watcher.
A periodic per-repository comparison between the in-memory queue and the
cluster would repair drift from any cause, including the deliberate case where
the reconciler keeps a slot because it could not verify a patch outcome (that
choice keeps the limit safe at the cost of capacity, and currently only a
restart gets the capacity back).
This is a safety net behind the targeted fixes, not a replacement for them.
It costs extra API calls, and the swap has to be done carefully: replace a
repository's queue atomically, and account for runs that leave the pending
state while the comparison is in flight.
Suggested only after the related crash, leak and rebuild issues are fixed; it
may turn out unnecessary if those prove sufficient in practice.
🧪 Testing Strategy
- Unit tests
- End-to-end tests: inject drift, verify the resync repairs it
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing #2890 and the related crash, leak, and rebuild work, then trace the watcher and reconciler behavior around the in-memory queue. Define how a per-repository comparison handles runs changing state during comparison and atomically replaces the queue. Add unit tests and an end-to-end test that injects drift and verifies resynchronization repairs it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100