Make sync processes compatible with autoscaling
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 30
- Forks
- 13
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 8
Description
Modern cloud infrastructures can auto scale compute resources to reduce costs and reap other benefits.
Accounts are currently assigned to a specific host by setting its sync_host property to the hostname the process is running on. This makes it very difficult to change the underlying compute resource which is likely a Docker container in modern infrastructures. Ideally the sync process wouldn't have to know or care what host/container it was running on.
Two potential options:
- Leverage the existing
.desired_sync_host(see this script) to reassign accounts after a scaling event. - Remove the concept of
.sync_hostcompletely from accounts and have them dynamically assigned to running sync processes. Use something like a redis hashring to assign accounts in an orderly fashion.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with bin/set-desired-host.py and trace how accounts use sync_host and desired_sync_host during sync assignment. Compare reassignment after scaling with dynamic assignment via a Redis hashring; done means sync processes can scale or move hosts without manual host-specific account assignment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, redis
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100