aws / aws/amazon-sagemaker-examples
scikit_bring_your_own fails to scale
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
Hi,
I am unable to scale the scikit_bring_your_own code across multiple instances when working with larger datasets. I am seeing this error when launching a batch transform job with multiple instances:
`*8 upstream prematurely closed connection while reading response header from upstream`
I tried adjusting the timeout settings in `nginx.conf` but this did not resolve the issue. I would truly appreciate guidance on this issue.
Contributor guide
Research direction
Start with the scikit_bring_your_own code and its nginx.conf settings, then reproduce the batch transform job with multiple instances and a larger dataset. Done means identifying the cause of the upstream connection closure and confirming that the job scales without that error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook, nginx, scikit-learn
- Domain
- cloud, distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100