aws / aws/sagemaker-pytorch-inference-toolkit

Endpoint dependencies error

Open
#91 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
143
Forks
73
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
I am using a package called Farm to perform the training and the inference. The training works fine: I pass the requirements.txt and all the packages are correctly installed. When I try to deploy it, there is also no problem. But at the moment of inference, when it is required to import farm, I get an error showing that the package doesn't exist.

**To reproduce**
```
estimator = PyTorch(
base_job_name="imdb",
max_run= 60*30,
entry_point="train_aws.py",
source_dir="source",
framework_version="1.5",
py_version = 'py3',
instance_count=1,
role=role,
hyperparameters=hyperparameters,
instance_type= "ml.p3.2xlarge",
output_path="s3:///opt/ml/model", # the S3 path where model is outputted
)
```

```
estimator.fit(
{'training': '/'.join(training_input_path.split('/')[:-1])},
wait=True
)
```

```
model = estimator.create_model(role=role, entry_point='inference.py')
mdm = multidatamodel.MultiDataModel(name="Models",
model_data_prefix="s3:///opt/ml/model/",
model = model,
sagemaker_session=sess
)
predictor = mdm.deploy(initial_instance_count=1,instance_type='ml.t2.medium', endpoint_name="inference")
```

Note: when I create the model.tar.gz, I include 'requirements.txt' and 'inference.py' on code folder.

**Expected behavior**

Inference should be working, but the packages are not being correctly installed. I try to do it using brute force, including these lines on inference.py:
```
import subprocess
import sys
subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'farm'])
```

But then it says that another package is missing: ImportError: cannot import name 'rng_integers'

Is there a better way to install the packages I want inside the inference container?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the packaging and entry points shown in requirements.txt, train_aws.py, inference.py, and the model.tar.gz layout. Reproduce the training and endpoint deployment flow, then compare dependency installation during training with inference. Done means the endpoint imports Farm and its dependencies successfully without ad hoc installation.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, pytorch
Domain
backend, cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.