haskell-distributed / haskell-distributed/distributed-process
Distributed cluster management tool
Nobody has claimed this yet.
- Dominant language
- Haskell
- Stars
- 751
- Forks
- 99
- Avg merge
- 45m
- Merged PRs (30d)
- 2
Description
There are a few issues with Cloud Haskell today that I think are showstoppers. The three that come to mind are:
- Peer discovery must be hand-rolled.
distributed-process-simplelocalnetimproves somewhat on this, but multicast isn't supported in cloud environments such as AWS, Azure, and GCP; - A cluster must effectively run the same binary at this time to use
spawn. Cluster version upgrades must be hand-rolled; - Cluster management (e.g. resource provisioning) must be hand-rolled.
Inspired by other tools such as Keter, I propose the creation of a toolkit that addresses these issues all at once.
The toolkit would have two parts.
The first is an agent that runs as a separate Cloud Haskell Process on every Node. This agent has an understanding of the underlying infrastructure (from Cloud or other, more bespoke environments), such that it can do peer discovery (via e.g. DNS), as well as peer registration needed to be discovered.
The second part of this toolkit is a command-line tool that allows to manage a cluster. The command-line tool interacts with the node agents to:
- Deploy applications;
- Query status;
- Perform system migrations from version to version, using the attached agent for inter-version communication;
I think that having a toolkit like this would be a huge improvement for productivity, and remove much of the friction in getting started with Cloud Haskell.
I am putting this on my (ever growing) list of things to do, but if someone is really keen on getting this to work, I'm happy to provide support!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the Cloud Haskell Process and Node concepts, then examine how distributed-process-simplelocalnet handles peer discovery. Define the scope of the proposed agent and command-line tool around peer discovery, deployment, status queries, and migrations; done requires agreement on the toolkit design before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell
- Domain
- cli, cloud, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100