Rewrite Allocator
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 3.7k
- Forks
- 676
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 6
Description
The Allocator code is bad. I'd hazard to say it's irredeemably bad. It's a source of constant bugs and breakages. Things are allocated twice or not allocated at all. The issues don't seem to be present (usually) in the levels deeper than swarmkit (libnetwork). Instead, libnetwork tends to give garbage responses when given garbage input.
Some examples of problems:
- A bunch of code for handling multiple allocators, even though only 1 allocator exists and no other allocators are anywhere in the roadmap
- Responsibilities muddled. All of the network allocation methods are just implemented straight on the Allocator object, even thought presence of a allocActor type in the Run function suggests they should be separate.
- Code is under-commented and tangled, making parsing it often inscrutable. The tangled nature of the code makes piece-wise refactoring almost impossible, because of the fragile nature of the code.
- The network allocator keeps a local state from which it makes decisions, which is supposed to be a mirror of the state as committed into raft. However, logic errors can cause this internal state to become irrecoverably inconsistent. If we're lucky, a leadership change fixes this before it becomes a problem. If we're not, bad local state is used to make bad decisions and create inconsistent distributed state. We need to reduce the footprint of the local state as much as feasible.
- The allocator requires the local state to be initialized from the distributed state before performing any new allocations. However, the logic for allocation and initialization follow the same code paths, and use a boolean flag to separate them. Errors in the initialization logic cause allocations and deallocations to occur before the local state is fully initialized, resulting in duplicate IP addresses.
We should rewrite the whole thing from scratch. It's not a small project, and there's a lot of risk in a rewrite versus a refactoring. However, a clean slate would let us escape the most ingrained design flaws.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by mapping the Allocator and allocActor entry points in Run, including the allocation paths shared with initialization and the local state mirrored into raft. A complete rewrite would need to separate allocator responsibilities, reduce fragile local state, and prevent duplicate or missing allocations, but the issue does not name specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems, networking
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100