open-compass / open-compass/VLMEvalKit
internvl3-1b model test
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 768
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 17
Description
/cloudml-dev-largeml-4/root/largelm-dev4/dl_work/songchen1/code/VLMEvalKit/vlmeval/vlm/internvl/internvl_chat.py:225: UserWarning: Following kwargs received: {'do_sample': False, 'max_new_tokens': 4096, 'top_p': None}, will use as generation config.
warnings.warn(f'Following kwargs received: {self.kwargs}, will use as generation config. ')
[2025-06-06 08:47:20] ERROR - RUN - run.py: main - 510: Model InternVL3-1B x Dataset MMMB_ar combination failed: str expected, not NoneType, skipping this combination.
Traceback (most recent call last):
File "/cloudml-dev-largeml-4/root/largelm-dev4/dl_work/songchen1/code/VLMEvalKit/run.py", line 382, in main
model = infer_data_job(
^^^^^^^^^^^^^^^
File "/cloudml-dev-largeml-4/root/largelm-dev4/dl_work/songchen1/code/VLMEvalKit/vlmeval/inference.py", line 192, in infer_data_job
model = infer_data(
^^^^^^^^^^^
File "/cloudml-dev-largeml-4/root/largelm-dev4/dl_work/songchen1/code/VLMEvalKit/vlmeval/inference.py", line 124, in infer_data
os.environ['WORLD_SIZE'] = ws_bak
~~~~~~~~~~^^^^^^^^^^^^^^
File "", line 719, in setitem
File "", line 793, in encode
TypeError: str expected, not NoneType
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in vlmeval/inference.py at line 124, then inspect the caller in run.py and the InternVL warning at vlmeval/vlm/internvl/internvl_chat.py:225. Reproduce the InternVL3-1B with MMMB_ar combination and determine why WORLD_SIZE is restored as None; done means the combination no longer fails with the reported TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100