ROCm/HIP Windows wheel: Translator destruction hangs indefinitely, even in CPU-only mode with no AMD GPU present

Summary

Destroying a ctranslate2.Translator (Python API) never returns when using the official ROCm/HIP Windows wheel — even with device="cpu" and zero GPU/HIP compute ever performed. The exact same code, same model, same ctranslate2 version (4.8.2), with the standard CUDA/CPU wheel from PyPI instead, destroys the object in ~15ms.

This reproduces on a machine with no AMD GPU at all. The only AMD-specific thing involved is that the ROCm wheel's ctranslate2.dll links against amdhip64_7.dll / hipblas.dll / rocblas.dll, so those get loaded into the process the moment ctranslate2 is imported — regardless of whether device="cuda"/"hip" is ever requested afterwards.

Environment

• OS: Windows 11 (build 10.0.26200)
• ctranslate2 version: 4.8.2 (both wheels)
  • Control: PyPI wheel (pip install ctranslate2==4.8.2)
  • Repro: official ROCm/HIP Windows wheel from the CTranslate2 v4.8.2 GitHub release (rocm-python-wheels-Windows.zip)
• HIP SDK: AMD ROCm 7.1 for Windows (C:\Program Files\AMD\ROCm\7.1\bin)
• GPU: NVIDIA GeForce RTX 4070 (no AMD GPU present on this machine)
• Python: 3.10.11

Minimal reproduction

import gc
import time
import ctranslate2

# Any CTranslate2-converted translation model works, e.g.:
#   pip install ctranslate2 transformers sentencepiece
#   ct2-transformers-converter --model Helsinki-NLP/opus-mt-en-fr \
#       --output_dir opus-mt-en-fr-ct2
MODEL_DIR = "opus-mt-en-fr-ct2"

t0 = time.monotonic()
translator = ctranslate2.Translator(MODEL_DIR, device="cpu", compute_type="float32")
print(f"created in {time.monotonic() - t0:.3f}s")

# any real call works fine
translator.translate_batch([["▁Hello", "</s>"]])

print("destroying...", flush=True)
t1 = time.monotonic()
del translator
gc.collect()
print(f"destroyed in {time.monotonic() - t1:.3f}s")  # <- never reached with the ROCm wheel
print("done")

Expected

Same as with the PyPI CUDA/CPU wheel: the destructor returns in a few milliseconds.

Actual

• PyPI wheel (CUDA/CPU): Translator created in 0.265s, destroyed in 0.016s. Script exits normally.
• ROCm wheel (same version, same script, same model, same device="cpu"): Translator created in 0.250s (and a real translate_batch call succeeds), but the del / destructor call never returns. Killed after 30s (nothing suggests it would ever return on its own — same process left running for minutes in earlier testing before being force-killed).

A py-spy dump of the hung process shows the Python frame permanently stuck at the del translator line (i.e. inside the native destructor call, never returning control to the interpreter):

Thread NNNNN (idle): "MainThread"
    main (repro.py:LINE)
        Locals: ... translator already deleted from locals, still inside native call

I also tried calling Translator.unload_model() explicitly before del in case it forces a synchronization/cleanup the destructor is missing — it returns immediately (0.016s) but does not avoid the subsequent hang on del, so it only frees the model weights, not whatever the destructor gets stuck on.

Why I think this matters beyond a hardware-specific edge case

I initially assumed this was specific to running actual inference on real AMD hardware. It is not — it reproduces on CPU-only compute, on a machine with no AMD GPU, with a real model loaded and used. The only thing that differs between the two wheels is that the ROCm one has the HIP runtime DLLs loaded in the process.

This matches a pattern already reported independently in other projects on ROCm+Windows:

• pytorch/pytorch#160759 — PyTorch+ROCm processes on Windows hang indefinitely on exit, even with minimal code (torch.randn(1,1,device="cuda") without touching the result). Resolved/worked around by calling torch.cuda.synchronize() (or anything that forces a sync) before exit.
• ROCm/ROCm#3418 — plain HIP programs (no ML framework at all) hanging inside amdhip64.dll after leaving main().

That makes me think the root cause may live in the HIP runtime/driver itself rather than in CTranslate2's own code, but I wanted to report it here specifically since (a) I couldn't find an existing CTranslate2 issue describing this exact shutdown hang, and (b) CTranslate2's CUDA backend has had to fix a comparable class of bug before ("Free curand states before the thread is destroyed", v4.7.2) — so it's plausible the newer HIP backend (introduced in 4.7.0) is missing an equivalent explicit cleanup/sync step before the native object is destroyed, even if the underlying trigger is a HIP runtime quirk.

Context

We hit this in production while building an experimental ROCm variant of a desktop app (real-time speech translation) that creates/destroys ctranslate2.Translator instances as part of normal operation (language-pair switching). The app-level symptom was: the main window closes fine, but the process itself never exits and shows up as a zombie in Task Manager.

One more data point that might help narrow this down: our first attempt at an app-level mitigation called os._exit(0) after a timeout instead of trying to wait for the destructor. That did not work either — the process was still alive over 20 seconds later, with the main thread's Python frame stuck inside the os._exit(0) call itself (confirmed with py-spy dump). On Windows, os._exit() calls ExitProcess(), which (unlike POSIX _exit()) still walks every loaded DLL's DLL_PROCESS_DETACH notification before actually terminating — and it appears to be that detach routine inside amdhip64_7.dll that hangs, not something specific to Translator's own destructor logic. This lines up with ROCm/ROCm#3418's title almost exactly ("HIP programs hang inside amdhip64.dll after main").

We only got a working mitigation by calling TerminateProcess() directly (via ctypes.windll.kernel32), which — by design — skips DLL_PROCESS_DETACH entirely instead of trying to make the DLL exit cleanly. That's obviously a last-resort workaround, not a fix, since we don't control CTranslate2's or the HIP runtime's native code.

Happy to provide more logs/traces or test a patch if there's a lead on where the HIP-specific cleanup would need to happen.

Start by running the provided Python reproduction with the PyPI CUDA/CPU wheel and the ROCm/HIP Windows wheel, then compare shutdown behavior around Translator destruction and process exit. Review the linked PyTorch and ROCm issues for related Windows HIP hangs; done means CPU-only Translator destruction and normal process termination complete without hanging, while preserving translation behavior.