Kaggle / Kaggle/kaggle-environments

Transformers need to be updated to the latest version.

Open
#336 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
452
Forks
190
Avg merge
31m
Merged PRs (30d)
2

Description

## 🚀 Feature

Update Transformers to the latest version to support Llama 3.3 and all upcoming models.

### Motivation

The Llama 3.2 model works fine with Transformers version 4.44.2, but when using Llama 3.3, there is an issue loading the tokenizer. The error message generated is:

```
Exception Traceback (most recent call last)
in ()
1 # Load tokenizer
----> 2 tokenizer = AutoTokenizer.from_pretrained(model_name)

...
Exception: data did not match any variant of untagged enum ModelWrapper at line 1251003 column 3
```

This error does not occur when using Transformers version 4.48.0. This indicates that Llama 3.3 (and potentially other upcoming models) may require an updated version of Transformers to function correctly.

### Additional context

- Model: Llama 3.2 works fine with Transformers 4.44.2.
- Model: Llama 3.3 fails with Transformers 4.44.2, but works with 4.48.0.
- The error is related to tokenizer loading, suggesting that higher versions of Transformers are necessary for future models.

Contributor guide

Open the contributing guide

Research direction

Locate the project configuration that pins Transformers, then reproduce the tokenizer-loading failure with Llama 3.3 using the current version. Update the dependency to a version that supports the model and verify that the tokenizer loads without the reported error.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.