open-compass / open-compass/VLMEvalKit
关于llava-v1.5-7b复现
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 768
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 17
Description
你好！我在复现llava-v1.5-7b时的评测结果达不到排行榜上的结果，很多数据集都会差几个百分点，请问llava-v1.5-7b复现还需要设置什么参数吗？我把这个模型下载下来了liuhaotian/llava-v1.5-7b，然后把它替换vlmeval/config.py 中llava-v1.5-7b的模型地址。这是我的脚本 CUDA_VISIBLE_DEVICES=1,2 python /VLMEvalKit/run.py --data MMBench_DEV_EN --model llava_v1.5_7b --verbose
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with vlmeval/config.py and the run.py entry point used by the provided MMBench_DEV_EN command. Compare the configured llava-v1.5-7b model path and evaluation settings with the reported leaderboard run, then reproduce the score difference. Done means identifying and documenting the setting or reproduction difference, if one can be confirmed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100