Lightning-AI / Lightning-AI/lightning-thunder
Tests incompatible with recent transformers
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
The following tests will be disabled with the bump of transformers from 4.52.4 to 4.55.4 and beyond and need fixing:
## ThunderFX being incompatible
- [x] thunder/tests/test_networks.py::test_hf_for_nemo
- [x] thunder/tests/test_networks.py::test_hf_phi3_vision
- [x] thunder/tests/test_networks.py::test_thunderfx_mistral_nemo_small
## thunder.jit being incompatible
- [ ] thunder/tests/test_networks.py::test_hf_kvcache (should be fixable by using the mask creation lookaside from the recipe)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with thunder/tests/test_networks.py::test_hf_kvcache and review the mask creation lookaside from the recipe. Check the test against transformers 4.55.4 or later, then confirm the test passes and no longer needs to be disabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100