Add Support for Latest Transformers Library in SageMaker
- Dominant language
- Python
- Stars
- 24
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the bug**
The current SageMaker SDK does not support Hugging Face `transformers` 4.38.0 and later, preventing the use of the latest models and features. This impacts users who need to deploy models like Qwen2.5-VL-7B-Instruct.
**To reproduce**
```python
import sagemaker
import boto3
from sagemaker.huggingface import HuggingFaceModel
try:
role = sagemaker.get_execution_role()
except ValueError:
iam = boto3.client('iam')
role = iam.get_role(RoleName='sagemaker_execution_role')['Role']['Arn']
# Hub Model configuration. https://huggingface.co/models
hub = {
'HF_MODEL_ID':'Qwen/Qwen2.5-VL-3B-Instruct',
'HF_TASK':'image-text-to-text'
}
# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
transformers_version='4.38.0',
pytorch_version='2.1.0',
py_version='py310',
env=hub,
role=role,
)
# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
initial_instance_count=1, # number of instances
instance_type='ml.m5.xlarge' # ec2 instance type
)
```
**Expected behavior**
Should work with latest version of transformers
**Screenshots or logs**
ValueError: Unsupported huggingface version: 4.38.0. You may need to upgrade your SDK version (pip install -U
sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.6.1, 4.10.2, 4.11.0, 4.12.3, 4.17.0,
4.26.0, 4.28.1, 4.37.0, 4.6, 4.10, 4.11, 4.12, 4.17, 4.26, 4.28, 4.37.

**Bug information**
- **SageMaker Core version**:
Contributor guide
Research direction
Start at the SageMaker HuggingFaceModel entry point and trace the validation that produces the unsupported huggingface version error. Compare the listed supported versions with the requested transformers 4.38.0 and later, then verify that the reported model configuration is accepted and covered by the relevant tests, if present.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, huggingface, python
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100