aws / aws/sagemaker-distribution

Set the inference server port based on SAGEMAKER_BIND_TO_PORT environment variable

Open
#645 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
154
Forks
120
Avg merge
5d 4h
Merged PRs (30d)
27

Description

### 🚀 The feature, motivation and pitch

Set the inference server port based on `SAGEMAKER_BIND_TO_PORT` environment variable, to support multi-container endpoints and inference pipelines.

If the container needs to listen on a second port, choose a port in the range specified by the SAGEMAKER_SAFE_PORT_RANGE environment variable.

Reference:
https://docs.aws.amazon.com/sagemaker/latest/dg/multi-container-troubleshooting.html#multi-container-missing-accept

### Use case description

_No response_

### Alternatives

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the AWS SageMaker multi-container troubleshooting reference linked in the issue and trace how the inference server currently chooses its port. Verify behavior for SAGEMAKER_BIND_TO_PORT and the SAGEMAKER_SAFE_PORT_RANGE second-port case; done means the container listens on the requested port and selects a safe port when needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker
Domain
backend, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.