aws-samples / aws-samples/sagemaker-hosting
Using ModelBuilder with Triton BLS and custom python environment
- Dominant language
- Jupyter Notebook
- Stars
- 46
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
I came across this today and I'm wondering if it's possible to use ModelBuilder to build a Triton model that uses BLS (i.e. the python backend). Specifically I currently have 2 steps, the first uses `conda-pack` to create a python 3.10 environment containing the python dependencies required by my model. The next step takes the `model.tar.gz` file from the train step, and the `pyenv.tar.gz` file created by `conda-pack` and builds a triton compatible model.
The example doesn't specify which triton backend to use, I guess it's torch?
https://github.com/aws-samples/sagemaker-hosting/blob/main/SageMaker-Model-Builder/traditional-models/model-builder-triton.ipynb
Contributor guide
Assessment
This issue has not been assessed yet.