perf(ROCm): parallel random sample with shared primitives
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: GFX1100-TG200
Replaces the serial <<<n,1>>> random-sample kernel with a block-cooperative argmax reduction over Gumbel scores, mirroring the CUDA fix from #1984. The RNG and argmax reduce come from include/vt/sample_common.h, shared across CPU/CUDA/ROCm so bit-identity is a build property, not a copy-sync property. The serial path is retained behind VT_FAST_RANDOM_SAMPLE=0 for same-binary A/B gating.
Adds ROCm + CUDA subprocess A/B test cases that re-exec the binary with VT_FAST_RANDOM_SAMPLE=0 and =1 and assert byte-identical token ids across widths up to 248320 (Qwen3.8-27B vocab).
Verified: test_sampler 21/21 pass, 114 assertions. Token-identical to upstream baseline on Qwen3.5-4B Q4_K.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with include/vt/sample_common.h and locate the random-sample kernel and test_sampler entry points. Compare the serial and block-cooperative paths, then run the ROCm and CUDA subprocess A/B cases with VT_FAST_RANDOM_SAMPLE=0 and =1. Done means test_sampler passes and token IDs remain byte-identical across the stated vocabulary widths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100