aws / aws/sagemaker-pytorch-training-toolkit

Worker initialization

Open
#115 8 comments 0 reactions 0 assignees View on GitHub
type: bug
Dominant language
Python
Stars
202
Forks
96
PR merge metrics
No merged PRs in 30d

Description

It seems like there is a bug in initialization logic.
Gunicorn processes are initialized not at container start but at a time of first request arrival.
The global `app` variable [here](https://github.com/aws/sagemaker-pytorch-container/blob/master/src/sagemaker_pytorch_container/serving.py#L101
) is not shared between gunicorn processes, so each process will be initialized only at a request arrival.

This will cause a random behavior. If the request comes to a worker that was already initialized - it will be processed quickly. If the request comes to a worker that is not yet initialized - the response will be delayed for quite some time (>30 sec in my case).
This will even cause `/ping` requests to time out and inability to deploy a container to AWS.

Contributor guide

Open the contributing guide

Research direction

Start by reading the initialization logic around src/sagemaker_pytorch_container/serving.py:101 and trace how workers are started in the container. Reproduce the delayed initialization and /ping timeout behavior if possible; done means workers initialize at container start rather than on their first request, without delayed or timing-out health checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, python
Domain
backend, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.