aws / aws/amazon-sagemaker-examples
ONNX inference with EI example does not actually use EI
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Link to the notebook**
https://github.com/aws/amazon-sagemaker-examples/blob/master/sagemaker-python-sdk/mxnet_onnx_eia/mxnet_onnx_eia.ipynb
Especially:
https://github.com/aws/amazon-sagemaker-examples/blob/master/sagemaker-python-sdk/mxnet_onnx_eia/resnet152.py
**What aspects of the notebook can be improved?**
EI is not used during inference (prediction time is same whether you attach EI or not).
**What are your suggestions?**
Replace:
```
mod = mx.mod.Module(symbol=sym, data_names=["data"], label_names=None)
```
With (inspired by https://github.com/aws/sagemaker-mxnet-inference-toolkit/blob/master/src/sagemaker_mxnet_serving_container/default_inference_handler.py#L72):
```
INFERENCE_ACCELERATOR_PRESENT_ENV = 'SAGEMAKER_INFERENCE_ACCELERATOR_PRESENT'
context = mx.cpu()
if os.environ.get(INFERENCE_ACCELERATOR_PRESENT_ENV) == 'true':
context = mx.eia()
mod = mx.mod.Module(symbol=sym, data_names=["data"], label_names=None, context=context)
```
Contributor guide
Research direction
Open sagemaker-python-sdk/mxnet_onnx_eia/mxnet_onnx_eia.ipynb and especially resnet152.py; start by checking how the MXNet Module is initialized for inference. Verify the example selects the inference accelerator when the documented environment variable is present, and confirm the inference path uses EI rather than showing the same prediction time as CPU-only execution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook, python
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100