Batch inference: The representation results of a sample repeated k times and n times are different？

I tested it using the following code in demo.py
processor_inputs = { "text": ['<|image_pad|> Represent the given image.\n', '<|image_pad|> Represent the given image.\n', '<|image_pad|> Represent the given image.\n'], "images": [Image.open('assets/example.jpg'), Image.open('assets/example.jpg'), Image.open('assets/example.jpg')], } inputs = Qwen2_VL_process_fn( processor_inputs, processor) inputs = batch_to_device(inputs, "cuda") with torch.autocast(device_type="cuda", dtype=torch.bfloat16): tgt_output = model(tgt=inputs)["tgt_reps"] print(tgt_output)

The greater the gap between n and k, the greater the gap in the representation obtained through inference. Theoretically, without the influence of padding, the results should be consistent.
k=1

n=8

In the actual model inference process, due to the influence of padding within the batch, the calculated similarity differs significantly from the result of single-sample inference.

Start with the reproduction in demo.py, including Qwen2_VL_process_fn, batch_to_device, and the model(tgt=inputs) call; compare k=1 with n=8 under the shown autocast settings. Trace how batch padding reaches representation computation and verify that repeated identical samples produce consistent representations and similarity results within expected numerical tolerance.