Lightning-AI / Lightning-AI/lightning-thunder

enable HF transformers 4.43.

Open
#2,302 1 comment 0 reactions 0 assignees View on GitHub
enhancement hf-transformers interpreter recipe
Dominant language
Python
Stars
1.5k
Forks
121
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature

enable HF transformers 4.43

### Motivation

HuggingFace Transformers 4.43.0 started using vmap in a path we are hitting:

https://github.com/huggingface/transformers/blob/df12d87d184db59aed00b6b22c2daff7bac95204/src/transformers/masking_utils.py#L347-L351

### Pitch

Maybe we can check for the presence and lookaside the `recent_torch` function to use the `older_torch` one instead...

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the Transformers masking_utils.py section at lines 347-351 and the issue's recent_torch and older_torch paths. Reproduce the vmap-related failure with HuggingFace Transformers 4.43.0, then verify that Lightning Thunder supports this version without regressing older versions.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python, pytorch
Domain
compilers, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.