bug: [Benchmark] ValueError: Length mismatch in multi-round-qa benchmark DataFrame creation

Describe the bug

Bug Description

When running the multi-round-qa benchmark, the program crashes with a ValueError due to length mismatch between different lists when creating the DataFrame.

Error Details

ValueError: Length of values (2) does not match length of index (1)

Error Location

File: benchmarks/multi-round-qa/multi-round-qa.py
Line: 332 in UserSession.summary() method

df["ttft"] = self.ttfts

Root Cause

The issue occurs when creating a DataFrame in the UserSession.summary() method. Different lists (self.ttfts, self.prompt_lengths, etc.) have inconsistent lengths:

• self.ttfts length: 2
• self.prompt_lengths length: 1
• Other lists also have length 1

Steps to Reproduce

1. Run the multi-round-qa benchmark
2. Wait for the first summary to be generated
3. The error occurs when manager.summary() is called

Expected Behavior

All lists should maintain consistent lengths, and the DataFrame creation should succeed.

Additional Context

This appears to be related to asynchronous request handling where the _update_result method might be called multiple times for the same response, or there's a race condition in the async callback system.

To Reproduce

it is always happened to me

Expected behavior

No response

Additional context

No response

Start in benchmarks/multi-round-qa/multi-round-qa.py at UserSession.summary() around line 332, then trace manager.summary() and the asynchronous _update_result path. Reproduce the multi-round-qa benchmark and inspect why self.ttfts has a different length from self.prompt_lengths and the other lists. Done means the summary DataFrame is created without a length-mismatch ValueError and the lists remain consistent.