[Performance]: Can the batch scheduler of TensorRT-LLM schedule the inference request from a Scaffolding request together.

Proposal to improve performance

This is a discussion.
In theory, scheduling requests for the same agent task together as much as possible can help shorten the latency of the agent task.

The image is from this [paper](https://arxiv.org/abs/2412.20993)

Can we do that on Scaffolding?

Report of performance regression

No response

Misc discussion on performance

No response

Your current environment (if you think it is necessary)

System Information:

• OS:
• Python version:
• CUDA version:
• GPU model(s):
• Driver version:
• TensorRT version:
• PyTorch version:
• TensorRT-LLM version:

Detailed output:

Paste the output of the above commands here

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.