HIP: hipMallocAsync silently corrupts buffers on gfx1030 (RDNA2) although the device reports memory-pool support; default allocator should be cub_caching on HIP

The cub_caching workaround is already documented in #2012 and #1072. This report adds
a failure mode that has not been described yet, and the reason the default selection
cannot protect against it: on gfx1030 the default allocator does not crash. It
silently produces wrong output, and the device tells CTranslate2 that memory pools are
supported.

Environment

GPU	AMD Radeon RX 6950 XT, Navi 21, gfx1030 (RDNA2), 16 GB
ROCm / HIP	7.2.53211
CTranslate2	4.8.1, official rocm-python-wheels-Linux.zip from the release page
faster-whisper	1.2.1
PyTorch	2.13.0+rocm7.2 (imported first; it brings libamdhip64 and libhiprand into the process)
Python / OS	3.13.5 / Debian 13 (trixie), kernel 6.17
Model	Systran/faster-whisper-large-v3

No HSA_OVERRIDE_GFX_VERSION is needed, gfx1030 is a native build target.

Failure mode: silent corruption, not a crash

Unlike #2012 (RX 6600, immediate illegal memory access at model load), on this card the
model loads fine, inference runs, and the output looks superficially plausible. It is just
wrong: transcription silently loses 32% to 95% of the content, and the result differs on
every run of the same audio in the same process.

Test audio: 233 s of Polish speech, deliberately built as
60s speech | 3s silence | 45s speech | 10s silence | 45s speech | 25s silence | 45s speech.
Identical decoding parameters in every run (VAD on, temperature fallback, standard
anti-hallucination thresholds).

Configuration	Segments across runs	Words	Similarity to reference	Reproducible
GTX 1080, int8_float32 (reference)	39	414	100%	yes
Intel CPU, int8_float32	39	412	96.9%	yes
gfx1030, float16, default allocator	16 / 25 / 27 / 28	175-235	44-68%	no
gfx1030, int8_float16, default	17	122	38.4%	no
gfx1030, int8, default	6	52	18.5%	no
gfx1030, float32, default	10	41	13.6%	no
gfx1030, int8_float32, default	6	16	5.6%	no
gfx1030, float16, CT2_CUDA_ALLOCATOR=cub_caching	38 / 38 / 38	419	96.2%	yes

Compute type is irrelevant. Every precision is broken with the default allocator, every
precision is correct with cub_caching.

With the rejection thresholds disabled, one segment came back with:

avg_logprob = -291.129

The healthy range on the same audio is roughly -0.2 to -1.5. That is a corrupted
number, not model uncertainty.

AMD_SERIALIZE_KERNEL=3 does not change anything (still three different results in three
runs), so this is not a kernel race. The buffers are wrong before any kernel runs.

After setting CT2_CUDA_ALLOCATOR=cub_caching

On the same card and the same wheel:

• three consecutive runs produce a byte-identical transcript (same sha256 of the text)
• the output is byte-identical to the CPU float32 result from the same library
  version (eb0b0108beca1654 for both)
• 16.3 s for the 233 s file; the broken runs took 19-47 s, mostly because the temperature
  fallback kept re-decoding garbage windows
• VRAM usage drops from about 9 GB to 5.3 GB

Minimal reproduction

import os, hashlib
os.environ.setdefault("CT2_VERBOSE", "1")      # prints "Using CUDA allocator: ..."
import torch                                    # must come before faster_whisper on ROCm
from faster_whisper import WhisperModel

wm = WhisperModel("large-v3", device="cuda", compute_type="float16")
for i in range(3):
    segs, _ = wm.transcribe("speech.wav", language="pl", vad_filter=True,
                            condition_on_previous_text=False)
    text = " ".join(s.text for s in segs)
    print(i, len(text), hashlib.sha256(text.encode()).hexdigest()[:16])

Any speech file of two minutes or more shows it. Run once as is, once with
CT2_CUDA_ALLOCATOR=cub_caching. Default: three different hashes and lengths.
cub_caching: three identical hashes. I can share the test file on request.

Why the automatic selection cannot catch this

support_cuda_malloc_async() in src/cuda/allocator.cc asks the device. HIP API trace
of the actual process (AMD_LOG_LEVEL=4), default settings:

hipGetDeviceCount ( ... )              Returned hipSuccess
hipDeviceGetAttribute ( ..., 88, 0 )   Returned hipSuccess
[ctranslate2] [info] Using CUDA allocator: cuda_malloc_async
hipMallocAsync ( ..., 2560, ... )      Returned hipSuccess

Attribute 88 is hipDeviceAttributeMemoryPoolsSupported (same value in ROCm 6.2, 7.0,
7.1, 7.2 headers and in the header shipped with the PyTorch ROCm wheel). Probing the same
runtime directly:

attr 87 = 32   hipDeviceAttributeWarpSize              (gfx1030 is wave32, matches)
attr 88 = 1    hipDeviceAttributeMemoryPoolsSupported  <- the device claims support

So the selection logic works as written: the device says memory pools are supported,
CTranslate2 believes it, hipMallocAsync returns hipSuccess, and the buffers are
unusable. The HIP branch of CT2_USE_ASYNC_ALLOC (line 22, !_WIN32) has no gate for
architectures where the device's claim is false, while the CUDA branch (line 26) is at
least version-gated. #1072, #2012, #2021 and this report all concern consumer RDNA parts
and the same allocator.

I will report the runtime side (hipMallocAsync on gfx1030 returning corrupt buffers
although MemoryPoolsSupported = 1) to ROCm separately.

Suggested changes

1. Default to cub_caching on HIP builds, or gate cuda_malloc_async on HIP to
   architectures where it is known to work. cub_caching was faster on this card anyway
2. Log the queried capability next to the chosen allocator
   (Using CUDA allocator: cuda_malloc_async (device reports memory pools: 1)),
   so this class of problem is visible without a debugger

Related: #1072, #2012 (same workaround, crash instead of silent corruption), #2021
(gfx1201, crash).

Happy to run additional tests on this card.

---

Edited after tracing the HIP API calls: the first version of this report claimed that
hipDeviceGetAttribute returned 0 for memory-pool support. That was my mistake (wrong enum
value). The device reports 1, and the analysis above reflects the trace.

Start in src/cuda/allocator.cc, especially support_cuda_malloc_async and the CT2_USE_ASYNC_ALLOC HIP and CUDA branches around lines 22 and 26. Reproduce the gfx1030 behavior with the provided faster-whisper script using the default allocator and cub_caching, then inspect related issues #1072, #2012, and #2021. Done means the unsafe default path is addressed and allocator capability information is visible in the log.