aws / aws/sagemaker-huggingface-inference-toolkit

Task `image-text-to-text` fails with `AttributeError: 'str' object has no attribute 'pad_token_id'`

Open
#135 5 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
270
Forks
60
PR merge metrics
No merged PRs in 30d

Description

When using the task type `image-text-to-text`, the [`tokenizer` is set to `image_url`](https://github.com/aws/sagemaker-huggingface-inference-toolkit/blob/5a7519da0ba37895d9e07712124e820c62ec4e56/src/sagemaker_huggingface_inference_toolkit/transformers_utils.py#L272), resulting in `pipeline` being called with `tokenizer` as a string. This causes an [error within `transformers`](https://github.com/huggingface/transformers/issues/36731).

I'm unsure if this task should instead set `feature_extractor`, or just leave `tokenizer` as `None`.

## Suggested fix

1. Instead of manually determining which tasks require `feature_extractor` or `tokenizer`, is it possible to process the full list of supported tasks from `transformers`, and then add the correct value based on the class structure? This will make the code much more future proof as `transformers` updates.

2. Add an environment variable to set the tokenizer. This way, if a similar error occurs in the future, developers can do a quick fix by overriding the value. (It may be worth noting that I am using a pre-built docker container, so I don't have the ability to modify this myself for a quick fix without doing a lot of other work. For this situation, an environment variable would be ideal.)

Contributor guide

Open the contributing guide

Research direction

Start in src/sagemaker_huggingface_inference_toolkit/transformers_utils.py around line 272 and review how the image-text-to-text task passes values to the transformers pipeline. Read the linked transformers issue to understand the pad_token_id failure, then determine whether this task needs feature_extractor or no tokenizer and evaluate the proposed environment-variable override. Done means the task no longer passes a string as tokenizer and its intended pipeline configuration is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.