Visual foundation model as plugin of Vicuna
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
As one of the most competitive open-sourced & LLM-based dialogue agents, we always believe Vicuna and (other LLaMA-based models) deserve more efforts in extending their capability to "perceive" visual signals. Following this belief, we develop a project called "Video-LLaMA", which enables Vicuna-13B to understand video representations and answer simple questions regarding the given video.
Check our Video-LLaMA at https://github.com/DAMO-NLP-SG/Video-LLaMA and hope you guys will be interested.
P.S. Credit should also be given to the pioneering works MiniGPT-4, LLaVA, and mPLUG-Owl, which share the same thought with us.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the Video-LLaMA project linked in the issue and comparing its proposed visual capabilities with FastChat's current scope. The issue does not name FastChat files, entry points, tests, integration requirements, or completion criteria, so those would need to be established before implementation can begin.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100