ROCm / ROCm/rocm-libraries

[Issue]: [hipblasLt]: hipblasLtCreate() is calling exit() if hipMalloc fails on machines with no devices

Open
#8,917 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Assembly
Stars
431
Forks
407
Avg merge
3d 7h
Merged PRs (30d)
441

Description

Problem Description

Summary:

hipDNN unit tests are failing prematurely on machines with no GPU devices. This is not a GTest assertion failure. The process exits with code 1 from inside hipblasLtCreate(), after printing:

Hip error: 'no ROCm-capable device is detected'(100) at /TheRock/rocm-libraries/projects/hipblaslt/library/src/amd_detail/hipblaslt.cpp:164

Backtrace shows:

   hipblasLtCreate()
   hipblaslt_plugin::HipblasltHandleFactory::createHipblasltHandle()
   hipdnnEnginePluginCreateImpl()
   EnginePlugin::createHandle()
   EnginePluginResourceManager::EnginePluginResourceManager()
   hipdnnHandle::hipdnnHandle()
   hipdnnCreate()
   expectAdditionalBackendApiForwardsToBackend()
   DirectBackendWrapperAdditionalApiForwardsToBackend_Test::TestBody()

The test is trying to validate wrapper API forwarding, but hipdnnCreate eagerly initializes engine plugin handles. Loading/initializing the hipBLASLt plugin calls hipblasLtCreate, which hard-exits when no ROCm-capable device is available or /dev/kfd is inaccessible.

The decision on whether or not to exit() should be left to the consumer of the library.

Suggested Workaround:

Skip hipblasLt handle creation in hipDNN by checking device count and return a gentler error rather than calling exit()

Operating System

Ubuntu 24.04.4 LTS

CPU

AMD EPYC 9654 96-Core Processor

GPU

None

ROCm Version

7.14

ROCm Component

hipBLASLt

Steps to Reproduce

Call hipblasLtCreate() on a machine with no GPU.

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in library/src/amd_detail/hipblaslt.cpp at line 164 and reproduce hipblasLtCreate() on a machine without a GPU or accessible /dev/kfd. Trace how the no-device hipMalloc failure is handled, then verify that the library returns an error instead of exiting the process.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.