facebookresearch / facebookresearch/matrix
Support image/video in Inference Command
Open
enhancement
- Dominant language
- Python
- Stars
- 291
- Forks
- 44
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe:**
Currently matrix inference only support text.
**Describe the solution you would like:**
1. Find huggingface dataset contains images (eg cais/hle) and videos
2. Convert the row into Chat message
3. Deploy maverick-fp8 on h100 and test it on images, which should work.
4. Explore video input and model, eg gemma
**Additional Context:**
We currently integrated vllm v0.8.3, please see if it support video models.
Contributor guide
Assessment
This issue has not been assessed yet.