GCG推理时无法批处理
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 55
- PR merge metrics
- No merged PRs in 30d
Description
作者您好,我发现在我移植的gcg推理代码中无法实现批处理,具体如下:
predict, seg_hidden_states = forward_model(
batch_inputs, pixel_values,
self.omg_tokenizer, self.omg_model, self.omg_llm,
self.omg_projector_text2vision,
self.omg_gen_config, self.omg_stop_criteria)
对于batch为4的图像输入,pixel_values包含四个元素,如下:
但是predict, seg_hidden_statesg尺寸不对,是对应单图的尺寸,如下:
怀疑是调用代码的这一句有问题,当pixel_values为单个或多个元素的list时,mm['inputs_embeds']都为(1,68,4096)的向量
mm_inputs = prepare_inputs_labels_for_multimodal(llm=llm, input_ids=ids, pixel_values=pixel_values)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the GCG inference call to forward_model and the named prepare_inputs_labels_for_multimodal function. Reproduce the issue using four image inputs and inspect mm['inputs_embeds'], predict, and seg_hidden_states. Done means batched inputs produce outputs with the corresponding batch dimensions rather than single-image shapes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100