aws / aws/sagemaker-huggingface-inference-toolkit
SageMaker fails because Conversation object is not found
- Dominant language
- Python
- Stars
- 270
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
We are hosting a model in SageMaker and today we observed the following error in our logs when the model was being relaunched in the instance:
```
ImportError: cannot import name 'Conversation' from 'transformers.pipelines' (/opt/conda/lib/python3.9/site-packages/transformers/pipelines/__init__.py)
```
I found someone reported a similar issue in https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556. When digging into the changes in the `transformers` dependency I found this change regarding the `Conversation` object: https://github.com/huggingface/transformers/pull/31165
Based in our logs, the last time the model was successfully relaunched in our SageMaker infrastructure (2 days ago), the version of the package that was downloaded was `transformers-4.41.2-py3-none-any.whl`, but when the error started to be observed, the downloaded version was `transformers-4.42.1-py3-none-any.whl`.
According to the pull request mentioned above, the change is effective as per version `4.42`, which was released 18 hours ago at the moment of writing this issue.
I think a change is needed in `src/sagemaker_huggingface_inference_toolkit/transformers_utils.py` to reflect the new structure of the code in the `transformers` dependency or pin the dependency version to version `4.41.2` to prevent the issue in the future.
In our case, we are updating our `requirements.txt` file to pin the version `4.41.2`, but other users might not be aware of what is happening, therefore they are not aware of this fix.
Contributor guide
Research direction
Start by reading src/sagemaker_huggingface_inference_toolkit/transformers_utils.py and compare its Conversation import with the transformers 4.42.1 package structure. Reproduce the SageMaker model relaunch using the reported dependency versions; done means the toolkit loads without the ImportError while supporting the changed transformers dependency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100