aws / aws/sagemaker-huggingface-inference-toolkit
Task `image-text-to-text` fails with `AttributeError: 'str' object has no attribute 'pad_token_id'`
- Dominant language
- Python
- Stars
- 270
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
When using the task type `image-text-to-text`, the [`tokenizer` is set to `image_url`](https://github.com/aws/sagemaker-huggingface-inference-toolkit/blob/5a7519da0ba37895d9e07712124e820c62ec4e56/src/sagemaker_huggingface_inference_toolkit/transformers_utils.py#L272), resulting in `pipeline` being called with `tokenizer` as a string. This causes an [error within `transformers`](https://github.com/huggingface/transformers/issues/36731).
I'm unsure if this task should instead set `feature_extractor`, or just leave `tokenizer` as `None`.
## Suggested fix
1. Instead of manually determining which tasks require `feature_extractor` or `tokenizer`, is it possible to process the full list of supported tasks from `transformers`, and then add the correct value based on the class structure? This will make the code much more future proof as `transformers` updates.
2. Add an environment variable to set the tokenizer. This way, if a similar error occurs in the future, developers can do a quick fix by overriding the value. (It may be worth noting that I am using a pre-built docker container, so I don't have the ability to modify this myself for a quick fix without doing a lot of other work. For this situation, an environment variable would be ideal.)
Contributor guide
Research direction
Start in src/sagemaker_huggingface_inference_toolkit/transformers_utils.py around line 272 and review how the image-text-to-text task passes values to the transformers pipeline. Read the linked transformers issue to understand the pad_token_id failure, then determine whether this task needs feature_extractor or no tokenizer and evaluate the proposed environment-variable override. Done means the task no longer passes a string as tokenizer and its intended pipeline configuration is verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100