aws / aws/sagemaker-pytorch-inference-toolkit
how to use gpu in sagemaker instance
- Dominant language
- Python
- Stars
- 143
- Forks
- 73
- PR merge metrics
- No merged PRs in 30d
Description
hi i am going with a custom docker image with all the cuda cudnn installed and also tested locally gpu is being utilized. but when upload to ecr and create endpoint it does not create endpoint and says kindly make sure docker serve command is valid , from debugging i came to found out that inference toolkit is needed inside image for the image to see if sagemaker gpu is avail or not, but there is no sample dockerfile from which i can understand , kindly tell
1)how to enable cuda support in custom built docker images for sagemaker
2)will using prebuilt images e.g accountnum.aws.amazon.com/pytorch:1.10-cuda113-py3 directly use cuda/gpu of sagemaker instance?
Contributor guide
Research direction
Start with the custom Docker, ECR, and SageMaker endpoint flow described in the issue, then inspect this inference toolkit's serving expectations and the linked container build materials. Done means documenting how a custom image enables GPU support and whether the cited prebuilt PyTorch image uses the SageMaker GPU instance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker, python, pytorch
- Domain
- cloud, devops, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100