aws / aws/sagemaker-pytorch-inference-toolkit

renaming of mxnet-model-server in sagemaker-inference package 1.5.3 causing entrypoint with command `serve` to fail

Open
#88 1 comment 0 reactions 0 assignees View on GitHub
type: bug
Dominant language
Python
Stars
143
Forks
73
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
`sagemaker-inference` recently (10/15) released v1.5.3, which included [this commit](https://github.com/aws/sagemaker-inference-toolkit/commit/8efb1672798d747cd623e5dd2eb7919af87a1b80) updating the name of the model server artifact and command from `mxnet-model-server` to `multi-model-server`.

all containers defined in this repository install `sagemaker-inference` as a dependency of this repo itself, on lines

```dockerfile
RUN pip install --no-cache-dir "sagemaker-pytorch-inference<2"
```

and this repo's `setup.py` has an `install_requires` which includes `sagemaker-inference>=1.3.1`. as a result, `sagemaker-inference=1.5.3` installed.

so while the `Dockerfile`'s `CMD` value (which calls `mxnet-model-server` directly) will succeed, attempts to use the `ENTRYPOINT` with `serve` as a build arg will fail with message:

```
Traceback (most recent call last):
File "/usr/local/bin/dockerd-entrypoint.py", line 22, in
serving.main()
File "/opt/conda/lib/python3.6/site-packages/sagemaker_pytorch_serving_container/serving.py", line 39, in main
_start_model_server()
File "/opt/conda/lib/python3.6/site-packages/retrying.py", line 49, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
File "/opt/conda/lib/python3.6/site-packages/retrying.py", line 206, in call
return attempt.get(self._wrap_exception)
File "/opt/conda/lib/python3.6/site-packages/retrying.py", line 247, in get
six.reraise(self.value[0], self.value[1], self.value[2])
File "/opt/conda/lib/python3.6/site-packages/six.py", line 703, in reraise
raise value
File "/opt/conda/lib/python3.6/site-packages/retrying.py", line 200, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File "/opt/conda/lib/python3.6/site-packages/sagemaker_pytorch_serving_container/serving.py", line 35, in _start_model_server
model_server.start_model_server(handler_service=HANDLER_SERVICE)
File "/opt/conda/lib/python3.6/site-packages/sagemaker_inference/model_server.py", line 94, in start_model_server
subprocess.Popen(multi_model_server_cmd)
File "/opt/conda/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/opt/conda/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'multi-model-server': 'multi-model-server'

```

**To reproduce**
1. build any container
1. mount a model and `inference.py` (e.g. `half_plus_three`) into `/opt/ml/model`
1. `docker run [tag name] serve`

**Expected behavior**
tensorflow serving serves the mounted model / `inference.py`

**System information**
A description of your system. Please provide:
- **Toolkit version**: 2.0.5, but should apply to all versions
- **Framework version**: 1.4, but should apply to all versions
- **Python version**: 3.7
- **CPU or GPU**: cpu, but should apply to both
- **Custom Docker image (Y/N)**: N

Contributor guide

Open the contributing guide

Research direction

Start with the repository's Dockerfiles and setup.py, focusing on the dependency installation and the CMD/ENTRYPOINT paths used by the serve argument. Reproduce the failure by building a container, mounting a model and inference.py at /opt/ml/model, and running docker run [tag name] serve. Done means the mounted model and inference.py are served without the missing-command error.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.