modelscope / modelscope/ms-swift

关于多图 grounding 的任务

Open
#9,906 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

Checklist / 检查清单
  • I have searched existing issues, and this is a new question or discussion topic. / 我已经搜索过现有的 issues,确认这是一个新的问题与讨论。
Question Description / 问题描述

说明文档中有 关于 多图grounding任务数据 训练结构 的说明如下:
image_id: 通常用于多图grounding任务。该参数只有当bbox_type为'real'时生效,代表bbox对应的图片是第几张,用于缩放bbox。索引从0开始,默认全为第0张。image_id的数量需要和bbox的数量一致。例如:若bbox的长度为10,images的长度为2,那么image_id的长度需要是10,其值需要在{0, 1}集合内。
有2个问题:
1. 这个image_id 在给到模型训练的时候是怎么根据 id 0,1,2 和原始的图像列表 进行的对应
2. 推理的时候怎样正确解析。或者可以将每个box 和具体 输入的多张图的具体哪一张对应

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the multi-image grounding data documentation referenced in the issue, then trace how image_id is consumed during training and inference. Document how indices map to the input image list and how each bounding box is associated with its image; done means both questions have an unambiguous, reproducible answer.

Written by the indexing model from the issue text.

Assessment

Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.