aws / aws/sagemaker-python-sdk

Multi-container endpoint SKLearn support unclear

Open
#4,555 2 comments 1 reaction 1 assignee Claimed by @mollyheamazon View on GitHub
component: hosting type: feature request
Dominant language
Python
Stars
2.3k
Forks
1.3k
Avg merge
1d 22h
Merged PRs (30d)
35

Description

**What did you find confusing? Please describe.**

I am interested in using a multi-container endpoint, as described in AWS docs [here](https://docs.aws.amazon.com/sagemaker/latest/dg/multi-container-endpoints.html). Specifically, I am interested in using [`Direct` invocation mode](https://docs.aws.amazon.com/sagemaker/latest/dg/multi-container-direct.html).

In their documentation, as part of creating the multi-container endpoint, they call `sm_client.create_model` with a `Containers` argument:

```python
container1 = {
'Image': '123456789012.dkr.ecr.us-east-1.amazonaws.com/myimage1:mytag',
'ContainerHostname': 'firstContainer'
}

container2 = {
'Image': '123456789012.dkr.ecr.us-east-1.amazonaws.com/myimage2:mytag',
'ContainerHostname': 'secondContainer'
}
inferenceExecutionConfig = {'Mode': 'Direct'}

import boto3
sm_client = boto3.Session().client('sagemaker')

response = sm_client.create_model(
ModelName = 'my-direct-mode-model-name',
InferenceExecutionConfig = inferenceExecutionConfig,
ExecutionRoleArn = role,
Containers = [container1, container2]
)
```

In sagemaker-python-sdk with SKLearn, the equivalent code would seem to be [the `SKLearnModel` constructor](https://sagemaker.readthedocs.io/en/stable/frameworks/sklearn/sagemaker.sklearn.html#scikit-learn-model) and [its `deploy` function](https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.model.Model.deploy).
Neither of those seem to accept information on multiple containers.

Am I missing the function/input I'm supposed to use? Is there some different `SKLearnModelMultiContainer` class I'm supposed to be using instead? Am I supposed to use a combination of sagemaker-python-sdk and boto3's SageMaker client? Are multi-container endpoints just not supported with sagemaker-python-sdk yet?

I also tried searching [the docs site](https://sagemaker.readthedocs.io/) for "multi container" and variants of the phrase, and couldn't find anything useful.

**Describe how documentation can be improved**

Document *whether* multi-container endpoints are supported with sagemaker-python-sdk, and if they are, the way in which they're supported.

**Additional context**

I'm not the most knowledgeable with ML stuff, so maybe I'm totally misunderstanding something here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.