huggingface / huggingface/nanoVLM

VLMEvalKit support for nanoVLM

Open
#89 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5k
Forks
510
PR merge metrics
No merged PRs in 30d

Description

I can help to integrate nanoVLM into VLMEvalKit.

the goal is to enable evaluating the model built upon nanoVLM with benchmarks supported by VLMEvalKit.

ideally, this leads to the following format:

```python
# Load pretrained weights from Hub
from models.vision_language_model import VisionLanguageModel

model = VisionLanguageModel.from_pretrained("lusxvr/nanoVLM-222M")

# evalutation
cd path/to/VLMEvalKit/
python run.py --model lusxvr/nanoVLM-222M --data MME
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the `models.vision_language_model.VisionLanguageModel` entry point and the `run.py` command shown in the issue, then inspect how VLMEvalKit handles the MME benchmark. Done means the `lusxvr/nanoVLM-222M` pretrained model can be selected with `--model` and evaluated on MME.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.