aws / aws/sagemaker-pytorch-inference-toolkit

Support Elastic Inference for PyTorch 1.5.1

Open
#101 0 comments 9 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
143
Forks
73
PR merge metrics
No merged PRs in 30d

Description

Hello,

the inference-toolkit hasn't yet adopted the change required to use Elastic Inference with PyTorch 1.5.1. Following [this guide](https://docs.aws.amazon.com/elastic-inference/latest/developerguide/ei-pytorch-using.html). It looks like the `default_model_fn` would have to attach the `EIA` to the model before returning. Similar to [this ](https://github.com/TheissHe/sagemaker-pytorch-inference-toolkit/commit/b963649c125ff9f8e43ec22544f0cda209e2088e).

Any feedback is appreciated.

Thank you,
Theiss

Contributor guide

Open the contributing guide

Research direction

Start at the default_model_fn entry point and read the AWS Elastic Inference guide for PyTorch 1.5.1, then compare the linked reference commit. Done means the inference toolkit supports attaching Elastic Inference to models returned by default_model_fn and behaves correctly with PyTorch 1.5.1.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, pytorch
Domain
cloud, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.