aws / aws/sagemaker-core

Add Support for Latest Transformers Library in SageMaker

Open
#241 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
24
Forks
25
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
The current SageMaker SDK does not support Hugging Face `transformers` 4.38.0 and later, preventing the use of the latest models and features. This impacts users who need to deploy models like Qwen2.5-VL-7B-Instruct.

**To reproduce**
```python
import sagemaker
import boto3
from sagemaker.huggingface import HuggingFaceModel

try:
role = sagemaker.get_execution_role()
except ValueError:
iam = boto3.client('iam')
role = iam.get_role(RoleName='sagemaker_execution_role')['Role']['Arn']

# Hub Model configuration. https://huggingface.co/models
hub = {
'HF_MODEL_ID':'Qwen/Qwen2.5-VL-3B-Instruct',
'HF_TASK':'image-text-to-text'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
transformers_version='4.38.0',
pytorch_version='2.1.0',
py_version='py310',
env=hub,
role=role,
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
initial_instance_count=1, # number of instances
instance_type='ml.m5.xlarge' # ec2 instance type
)
```

**Expected behavior**
Should work with latest version of transformers

**Screenshots or logs**
ValueError: Unsupported huggingface version: 4.38.0. You may need to upgrade your SDK version (pip install -U
sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.6.1, 4.10.2, 4.11.0, 4.12.3, 4.17.0,
4.26.0, 4.28.1, 4.37.0, 4.6, 4.10, 4.11, 4.12, 4.17, 4.26, 4.28, 4.37.

![Image](https://github.com/user-attachments/assets/c8618d00-b08e-4c0b-b7fc-63dbcc8d88f1)

**Bug information**
- **SageMaker Core version**:

Contributor guide

Open the contributing guide

Research direction

Start at the SageMaker HuggingFaceModel entry point and trace the validation that produces the unsupported huggingface version error. Compare the listed supported versions with the requested transformers 4.38.0 and later, then verify that the reported model configuration is accepted and covered by the relevant tests, if present.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, huggingface, python
Domain
backend, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.