open-compass / open-compass/VLMEvalKit
关于AI2D评测是否有mask的问题问题
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 768
- Avg merge
- 2d 27m
- Merged PRs (30d)
- 18
Description
我在论文里https://arxiv.org/abs/2412.05271这篇论文里注意到AI2D有两个评价分数，分别是w M（with mask） 和 wo M（without mask）。
请教一下， internvl模型在AI2D上的评测是否采用和vlmevalkit同样的方法，在图片中加入选项的字母？这种方式对应with mask 和 without mask的哪一种？
感谢对开源工作的贡献，期待解答。
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the AI2D evaluation handling in VLMEvalKit and compare it with the method described in the linked paper. Determine whether adding option letters to images is used and whether it corresponds to the with-mask or without-mask score. Done means documenting a clear answer for InternVL and the toolkit’s evaluation behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100