aws / aws/amazon-sagemaker-examples

[Bug Report]: shm_size issue while deploying ensemble models on triton

Open
#3,506 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

**Describe the bug**
I am deploying an ensemble of an NLP model. While running the code specified, I get this error:
```
Unable to initialize shared memory key 'triton_python_backend_shm_region_2' to requested size
```
Based on my investigation, each of the directories with python_backend, needs 64MB of shm. On the other hand, there isn't any option to change the shm_size of the container. Then, how we can solve the problem?

Contributor guide

Open the contributing guide

Research direction

No file or test is named; start by reproducing the ensemble deployment described in the report and inspect the container's shared-memory configuration around Triton Python backend initialization. Done means a verified way to deploy the ensemble without the shared-memory error, with the required configuration documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.