Batch Inference

Dears,

Thank you so much for sharing your great code!

I think providing an inference code that supports batching will be very useful, e.g., it will speed up the evaluation process significantly.

Thanks in advance!

The issue does not identify an inference file, entry point, or test. Start by locating the existing inference and evaluation paths, then determine how inputs are represented and whether batching is supported elsewhere. Done should mean a documented batching path that speeds evaluation while preserving current inference behavior, with tests covering batched inputs.

Written by the indexing model from the issue text.