aws / aws/sagemaker-pytorch-inference-toolkit

how to use gpu in sagemaker instance

Open
#126 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
143
Forks
73
PR merge metrics
No merged PRs in 30d

Description

hi i am going with a custom docker image with all the cuda cudnn installed and also tested locally gpu is being utilized. but when upload to ecr and create endpoint it does not create endpoint and says kindly make sure docker serve command is valid , from debugging i came to found out that inference toolkit is needed inside image for the image to see if sagemaker gpu is avail or not, but there is no sample dockerfile from which i can understand , kindly tell
1)how to enable cuda support in custom built docker images for sagemaker
2)will using prebuilt images e.g accountnum.aws.amazon.com/pytorch:1.10-cuda113-py3 directly use cuda/gpu of sagemaker instance?

Contributor guide

Open the contributing guide

Research direction

Start with the custom Docker, ECR, and SageMaker endpoint flow described in the issue, then inspect this inference toolkit's serving expectations and the linked container build materials. Done means documenting how a custom image enables GPU support and whether the cited prebuilt PyTorch image uses the SageMaker GPU instance.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, python, pytorch
Domain
cloud, devops, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.