aws / aws/sagemaker-pytorch-inference-toolkit
Endpoint dependencies error
- Dominant language
- Python
- Stars
- 143
- Forks
- 73
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the bug**
I am using a package called Farm to perform the training and the inference. The training works fine: I pass the requirements.txt and all the packages are correctly installed. When I try to deploy it, there is also no problem. But at the moment of inference, when it is required to import farm, I get an error showing that the package doesn't exist.
**To reproduce**
```
estimator = PyTorch(
base_job_name="imdb",
max_run= 60*30,
entry_point="train_aws.py",
source_dir="source",
framework_version="1.5",
py_version = 'py3',
instance_count=1,
role=role,
hyperparameters=hyperparameters,
instance_type= "ml.p3.2xlarge",
output_path="s3:///opt/ml/model", # the S3 path where model is outputted
)
```
```
estimator.fit(
{'training': '/'.join(training_input_path.split('/')[:-1])},
wait=True
)
```
```
model = estimator.create_model(role=role, entry_point='inference.py')
mdm = multidatamodel.MultiDataModel(name="Models",
model_data_prefix="s3:///opt/ml/model/",
model = model,
sagemaker_session=sess
)
predictor = mdm.deploy(initial_instance_count=1,instance_type='ml.t2.medium', endpoint_name="inference")
```
Note: when I create the model.tar.gz, I include 'requirements.txt' and 'inference.py' on code folder.
**Expected behavior**
Inference should be working, but the packages are not being correctly installed. I try to do it using brute force, including these lines on inference.py:
```
import subprocess
import sys
subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'farm'])
```
But then it says that another package is missing: ImportError: cannot import name 'rng_integers'
Is there a better way to install the packages I want inside the inference container?
Contributor guide
Research direction
Start by reviewing the packaging and entry points shown in requirements.txt, train_aws.py, inference.py, and the model.tar.gz layout. Reproduce the training and endpoint deployment flow, then compare dependency installation during training with inference. Done means the endpoint imports Farm and its dependencies successfully without ad hoc installation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, pytorch
- Domain
- backend, cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100