rapidsai / rapidsai/deployment
EC2-MNMG instructions do not connect to workers within NVIDIA AWS instances
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 15
- Forks
- 41
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 6
Description
Deploying an ec2-mnmg cluster setup does not properly connect the workers that were created (and can be shut down) properly.
https://docs.rapids.ai/deployment/nightly/cloud/aws/ec2-multi/#cluster-setup
This may be a network issue with NVIDIA security groups or an issue with dask-cloud provider, which is reportedly needs more resources. Found during 23.12 deployment testing. cc @aravenel @jacobtomlinson
Proposed solutions:
- Test with a more permission security group, closer to our customer's or average user
- If the issue persists after testing one, allocate resources to fix any bugs in dask cloud provider.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the EC2-MNMG cluster setup instructions at the linked deployment page and reproduce the worker connection failure in an NVIDIA AWS instance. Test the proposed security-group change first, then determine whether the dask-cloud-provider resource issue remains; done means workers connect correctly and can be shut down as described.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100