Containers may not be cleaned up after restart
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 3.7k
- Forks
- 676
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 6
Description
When the agent has a task removed from its assignment set, it first updates the database to remove the task assignment, then attempts removal of the container (closeManager):
If the daemon crashes between updating the database and removing the container, I think the container won't be removed. On startup, the agent walks the database of tasks, and deletes any entries not marked "assigned":
It doesn't start task managers for these, which would be needed to perform the container removal.
Would it be more correct to have closeManager update the database after calling tm.ctlr.Remove?
cc @stevvooe
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in agent/worker.go at the startup task cleanup and the closeManager path linked in the issue. Trace how database assignment removal and tm.ctlr.Remove are ordered, then verify the daemon-crash-and-restart scenario. Done means unassigned tasks cannot leave their containers behind after a restart.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100