huggingface / huggingface/nanoVLM
VLMEvalKit support for nanoVLM
- Dominant language
- Python
- Stars
- 5k
- Forks
- 510
- PR merge metrics
- No merged PRs in 30d
Description
I can help to integrate nanoVLM into VLMEvalKit.
the goal is to enable evaluating the model built upon nanoVLM with benchmarks supported by VLMEvalKit.
ideally, this leads to the following format:
```python
# Load pretrained weights from Hub
from models.vision_language_model import VisionLanguageModel
model = VisionLanguageModel.from_pretrained("lusxvr/nanoVLM-222M")
# evalutation
cd path/to/VLMEvalKit/
python run.py --model lusxvr/nanoVLM-222M --data MME
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the `models.vision_language_model.VisionLanguageModel` entry point and the `run.py` command shown in the issue, then inspect how VLMEvalKit handles the MME benchmark. Done means the `lusxvr/nanoVLM-222M` pretrained model can be selected with `--model` and evaluated on MME.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100