Lightning-AI / Lightning-AI/litgpt
Transformers version bump for recent model support
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 1.5k
- Avg merge
- 15h 37m
- Merged PRs (30d)
- 1
Description
Supporting recent models requires bumping transformers to >4.51, but this leads to issues as can be seen here: https://github.com/Lightning-AI/litgpt/pull/2029
related to deepspeed's use of pydantic.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the dependency changes and failure details in pull request 2029, then investigate the transformers upgrade and its interaction with deepspeed's pydantic usage. Done means recent models are supported with transformers above 4.51 without the reported compatibility issues.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100