Lightning-AI / Lightning-AI/litgpt
Feature: Support for Multimodality
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 1.5k
- Avg merge
- 15h 37m
- Merged PRs (30d)
- 1
Description
As my understanding (Please correct me if I'm wrong), currently LitGPT is not supporting Multimodality as the Pretrain, Finetune, Chat and Evals is not accepting any other modality inputs. I would like to help to support this if it's on the plan of improvements for LitGPT.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the existing pretrain, finetune, chat, and eval workflows to determine how they currently accept inputs. Define which modalities and model paths are in scope, then document completion criteria for multimodal support across those workflows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100