aws / aws/amazon-sagemaker-examples
The `serve` script in examples should use `sync`, not `gevent` workers
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
Hi,
We spent a lot of time on resolving an issue with some code based on the examples here, and we determined that the line here was stopping our code from behaving correctly when deployed via Sagemaker:
https://github.com/awslabs/amazon-sagemaker-examples/blob/d6a25381975332aa79df61144fc0c2421bd5168c/advanced_functionality/scikit_bring_your_own/container/decision_trees/serve#L51
When we replaced this with `sync` our code behaved correctly. This link has information about what might be the trouble: [Why You Should (Almost) Always Choose Sync Gunicorn Workers](https://hackernoon.com/why-you-should-almost-always-choose-sync-gunicorn-over-workers-ze9c32wj). Specifically,
> Using Gevent needs all of your code to be co-operative in nature or all libraries you use should be monkey-patchable. Now this line may sound simple but it’s a big ask. This means either ensuring all the DB drivers, clients, 3rd party libraries used all of these are either pure python, which would ensure they can be monkey-patched OR they are written specifically with Gevent support.
Given that the intent of the container/Sagemaker approach is to enable deployment of code written by data scientists and other people who may not be experienced enough to vet their chosen libraries for proper behavior under arcane asynchronous execution circumstances, I suggest changing the scripts provided here to use the more-likely-to-be-correct default of a `sync` worker.
Thanks!
Contributor guide
Research direction
Open advanced_functionality/scikit_bring_your_own/container/decision_trees/serve at line 51 and inspect the worker setting reported in the issue. Check whether other example serve scripts use the same setting; done means the affected SageMaker example defaults to sync workers instead of gevent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100