aws / aws/sagemaker-pytorch-inference-toolkit
Support Elastic Inference for PyTorch 1.5.1
- Dominant language
- Python
- Stars
- 143
- Forks
- 73
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
the inference-toolkit hasn't yet adopted the change required to use Elastic Inference with PyTorch 1.5.1. Following [this guide](https://docs.aws.amazon.com/elastic-inference/latest/developerguide/ei-pytorch-using.html). It looks like the `default_model_fn` would have to attach the `EIA` to the model before returning. Similar to [this ](https://github.com/TheissHe/sagemaker-pytorch-inference-toolkit/commit/b963649c125ff9f8e43ec22544f0cda209e2088e).
Any feedback is appreciated.
Thank you,
Theiss
Contributor guide
Research direction
Start at the default_model_fn entry point and read the AWS Elastic Inference guide for PyTorch 1.5.1, then compare the linked reference commit. Done means the inference toolkit supports attaching Elastic Inference to models returned by default_model_fn and behaves correctly with PyTorch 1.5.1.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, pytorch
- Domain
- cloud, machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100