aws / aws/sagemaker-huggingface-inference-toolkit
logs not showing up in cloudwatch
- Dominant language
- Python
- Stars
- 270
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
When deploying a question answering model with later versions of python, transformers and pytorch the logs do not seem to be seen in cloudwatch. If I deploy the same model but with older python and module versions the logs can be seen in cloudwatch.
# Recreate the issue
## Case 1, No logs seen
### Code
```
hub = {
'HF_MODEL_ID':'distilbert-base-uncased-distilled-squad',
'HF_TASK':'question-answering'
}
huggingface_model = HuggingFaceModel(
env=hub,
role=role,
transformers_version="4.12.3",
pytorch_version="1.9.1",
py_version="py38",
)
predictor = huggingface_model.deploy(
initial_instance_count=1,
instance_type="ml.m5.xlarge"
)
data = {
"inputs": {
"question": "What is used for inference?",
"context": "My Name is Philipp and I live in Nuremberg. This model is used with sagemaker for inference."
}
}
# request
predictor.predict(data)
```
### logs:
```
#015Downloading: 0%\| \| 0.00/2.74k [00:00
Contributor guide
Research direction
No repository file or test is named. Reproduce the two HuggingFaceModel deployments with the Python, PyTorch, and Transformers versions shown, then compare their SageMaker and CloudWatch output; done means logs from the newer environment are visible in CloudWatch as they are in the older environment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, pytorch
- Domain
- cloud, machine-learning, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100