aws / aws/sagemaker-huggingface-inference-toolkit

logs not showing up in cloudwatch

Open
#61 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
270
Forks
60
PR merge metrics
No merged PRs in 30d

Description

When deploying a question answering model with later versions of python, transformers and pytorch the logs do not seem to be seen in cloudwatch. If I deploy the same model but with older python and module versions the logs can be seen in cloudwatch.

# Recreate the issue

## Case 1, No logs seen

### Code
```

hub = {
'HF_MODEL_ID':'distilbert-base-uncased-distilled-squad',
'HF_TASK':'question-answering'
}

huggingface_model = HuggingFaceModel(
env=hub,
role=role,
transformers_version="4.12.3",
pytorch_version="1.9.1",
py_version="py38",
)
predictor = huggingface_model.deploy(
initial_instance_count=1,
instance_type="ml.m5.xlarge"
)

data = {
"inputs": {
"question": "What is used for inference?",
"context": "My Name is Philipp and I live in Nuremberg. This model is used with sagemaker for inference."
}
}

# request
predictor.predict(data)

```
### logs:

```

#015Downloading: 0%\| \| 0.00/2.74k [00:00

Contributor guide

Open the contributing guide

Research direction

No repository file or test is named. Reproduce the two HuggingFaceModel deployments with the Python, PyTorch, and Transformers versions shown, then compare their SageMaker and CloudWatch output; done means logs from the newer environment are visible in CloudWatch as they are in the older environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, pytorch
Domain
cloud, machine-learning, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.