DAMO-NLP-SG / DAMO-NLP-SG/VideoLLaMA3
Why only mlp2x_gelu pretrained weights when code supports multiple projector types?
- Dominant language
- Jupyter Notebook
- Stars
- 1.2k
- Forks
- 89
- PR merge metrics
- No merged PRs in 30d
Description
I notice that the VideoLLaMA3 codebase has support for multiple projector architectures:
mlp2x_gelu (currently used in all pretrained models)
mlp3x_gelu
linear
simp_spatial_conv
However, all the official released models (VideoLLaMA3-2B, VideoLLaMA3-7B, etc.) only use mlp2x_gelu projector.
Why were other projector types not trained and released as alternatives?
Are there plans to release variants with different projector types?
Can you share insights on the performance comparison between different projector types?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the projector architecture names listed in the issue: mlp2x_gelu, mlp3x_gelu, linear, and simp_spatial_conv. Done would require a maintainer-provided explanation of why released models use mlp2x_gelu, whether alternatives are planned, and how their performance compares; no files or tests are identified.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100