OpenGVLab / OpenGVLab/Ask-Anything

ValueError: too many values to unpack (expected 2) when testing Mvbench dataset with demo_mistral_hd.ipynb

Open
#263 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.4k
Forks
268
PR merge metrics
No merged PRs in 30d

Description

When I was testing the Mvbench dataset using demo_mistral_hd.ipynb based on mistral, I copied the test code from the mvbench.ipynb file and configured the dataset properly, but it showed such an error. Why is this?

`data_list = {
"Action Sequence": ("action_sequence.json", "/home/user/zhengxinye/datasets/mvbench/video/star/Charades_v1_480/", "video", True), # has start & end
"Action Prediction": ("action_prediction.json", "/home/user/zhengxinye/datasets/mvbench/video/star/Charades_v1_480/", "video", True), # has start & end
"Action Antonym": ("action_antonym.json", "/home/user/zhengxinye/datasets/mvbench/video/ssv2_video/", "video", False),
"Fine-grained Action": ("fine_grained_action.json", "/home/user/zhengxinye/datasets/mvbench/video/Moments_in_Time_Raw/videos/", "video", False),
"Unexpected Action": ("unexpected_action.json", "/home/user/zhengxinye/datasets/mvbench/video/FunQA_test/test/", "video", False),
"Object Existence": ("object_existence.json", "/home/user/zhengxinye/datasets/mvbench/video/clevrer/video_validation/", "video", False),
"Object Interaction": ("object_interaction.json", "/home/user/zhengxinye/datasets/mvbench/video/star/Charades_v1_480/", "video", True), # has start & end
"Object Shuffle": ("object_shuffle.json", "/home/user/zhengxinye/datasets/mvbench/video/perception/videos/", "video", False),
"Moving Direction": ("moving_direction.json", "/home/user/zhengxinye/datasets/mvbench/video/clevrer/video_validation/", "video", False),
"Action Localization": ("action_localization.json", "/home/user/zhengxinye/datasets/mvbench/video/sta/sta_video/", "video", True), # has start & end
"Scene Transition": ("scene_transition.json", "/home/user/zhengxinye/datasets/mvbench/video/scene_qa/video/", "video", False),
"Action Count": ("action_count.json", "/home/user/zhengxinye/datasets/mvbench/video/perception/videos/", "video", False),
"Moving Count": ("moving_count.json", "/home/user/zhengxinye/datasets/mvbench/video/clevrer/video_validation/", "video", False),
"Moving Attribute": ("moving_attribute.json", "/home/user/zhengxinye/datasets/mvbench/video/clevrer/video_validation/", "video", False),
"State Change": ("state_change.json", "/home/user/zhengxinye/datasets/mvbench/video/perception/videos/", "video", False),
"Fine-grained Pose": ("fine_grained_pose.json", "/home/user/zhengxinye/datasets/mvbench/video/nturgbd/", "video", False),
"Character Order": ("character_order.json", "/home/user/zhengxinye/datasets/mvbench/video/perception/videos/", "video", False),
"Egocentric Navigation": ("egocentric_navigation.json", "/home/user/zhengxinye/datasets/mvbench/video/vlnqa/", "video", False),
"Episodic Reasoning": ("episodic_reasoning.json", "/home/user/zhengxinye/datasets/mvbench/video/tvqa/frames_fps3_hq/", "frame", True), # has start & end, read frame
"Counterfactual Inference": ("counterfactual_inference.json", "/home/user/zhengxinye/datasets/mvbench/video/clevrer/video_validation/", "video", False),
}

data_dir = "/home/user/zhengxinye/datasets/mvbench/json"`

The error prompt is as follows:
`---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[67], line 14
12 acc_dict[task_type][1] += 1
13 total += 1
---> 14 pred = infer_mvbench(
15 example,
16 system="Carefully watch the video and pay attention to the cause and sequence of events, the detail and movement of objects, and the action and pose of persons. Based on your observations, select the best option that accurately addresses the question.\n",
17 question_prompt="\nOnly give the best option.",
18 answer_prompt="Best option:(",
19 return_prompt='(',
20 system_q=False,
21 print_res=True,
22 system_llm=True
23 )
24 gt = example['answer']
25 res_list.append({
26 'pred': pred,
27 'gt': gt
28 })

Cell In[65], line 19, in infer_mvbench(data_sample, system, question_prompt, answer_prompt, return_prompt, system_q, print_res, system_llm)
17 video_emb, _ = model.encode_img(video, system + data_sample['question'])
18 else:
---> 19 video_emb, _ = model.encode_img(video, system)
20 video_list.append(video_emb)
21 # video_list.append(torch.zeros_like(video_emb))

ValueError: too many values to unpack (expected 2)`

May I ask where the problem is? Please help solve it. Thank you!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in demo_mistral_hd.ipynb at infer_mvbench and compare its model.encode_img call with the corresponding code in mvbench.ipynb. Trace the returned values and update the notebook usage so the Mvbench evaluation runs without the unpacking error, then rerun the affected test cell.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.