Parallelize unfinished beam finalization across beams

Motivation

insertUnfinishedPathKernel launches a single CUDA thread and serially reconstructs every unfinished beam. At larger beam widths this creates a long serial tail during beam-search finalization, although reconstruction of distinct beams is independent.

Proposed change

Launch a 128-thread block, assign beams to threads, read the destination offset once through shared memory, synchronize, and update numBeamsCBA once after all beams are copied. Token reconstruction within each beam remains serial and preserves the current generated-suffix semantics.

Direct-kernel evidence

On an uncontended H100 (SM90, CUDA 13.0), beam width 80, generated length 5, nine repeats of 20,000 CUDA-event-timed launches after 1,000 warmups:

• current kernel median: 64.399017 us/call
• proposed kernel median: 5.086699 us/call
• delta: -92.1% (12.66x kernel speedup)

The baseline and candidate implementations were compiled into the same binary and their output buffers were compared before timing. This is isolated kernel evidence, not a claim of equal end-to-end request speedup.

Start at insertUnfinishedPathKernel and trace the beam-finalization call path. Preserve serial token reconstruction and generated-suffix semantics while assigning independent beams to threads, then verify completion by comparing the current and candidate output buffers and checking the final beam count update.