huggingface / huggingface/lighteval

Vision Language Model Support?

Open
#1,020 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
555
Avg merge
1d 6h
Merged PRs (30d)
1

Description

Hi I am using VLM but sometimes want to evaluate VLM on text only language dataset like MMLU or HumanEval or LiveCodeBench.

Is this repo supporting it?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.