aws / aws/amazon-sagemaker-examples

scikit_bring_your_own fails to scale

Open
#790 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Hi,

I am unable to scale the scikit_bring_your_own code across multiple instances when working with larger datasets. I am seeing this error when launching a batch transform job with multiple instances:

`*8 upstream prematurely closed connection while reading response header from upstream`

I tried adjusting the timeout settings in `nginx.conf` but this did not resolve the issue. I would truly appreciate guidance on this issue.

Contributor guide

Open the contributing guide

Research direction

Start with the scikit_bring_your_own code and its nginx.conf settings, then reproduce the batch transform job with multiple instances and a larger dataset. Done means identifying the cause of the upstream connection closure and confirming that the job scales without that error.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, nginx, scikit-learn
Domain
cloud, distributed-systems, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.