aws-samples / aws-samples/sagemaker-hosting

Using ModelBuilder with Triton BLS and custom python environment

Open
#20 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
46
Forks
13
PR merge metrics
No merged PRs in 30d

Description

I came across this today and I'm wondering if it's possible to use ModelBuilder to build a Triton model that uses BLS (i.e. the python backend). Specifically I currently have 2 steps, the first uses `conda-pack` to create a python 3.10 environment containing the python dependencies required by my model. The next step takes the `model.tar.gz` file from the train step, and the `pyenv.tar.gz` file created by `conda-pack` and builds a triton compatible model.

The example doesn't specify which triton backend to use, I guess it's torch?

https://github.com/aws-samples/sagemaker-hosting/blob/main/SageMaker-Model-Builder/traditional-models/model-builder-triton.ipynb

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.