iree-org / iree-org/wave

[SGLang] Wave runtime crashes when cache is turned off

Open
#459 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
59
Forks
32
PR merge metrics
No merged PRs in 30d

Description

As stated in the title, in SGLang environment paged decode attention kernels fails to launch when wave cache is turned off. Error log below:

```
> wave_runtime.launch(kernel_launch_info, kernel_args, dyn_dims, scalar_args)
E RuntimeError: Error 719(hipErrorLaunchFailure) /project/wave_lang/kernel/wave/runtime/runtime.cpp:172:
E hipModuleLaunchKernel( function, info.gridX, info.gridY, info.gridZ, info.blockX, info.blockY, info.blockZ, 0, stream, nullptr, (void **)&hipLaunchParams)
E unspecified launch failure

/opt/venv/lib/python3.10/site-packages/wave_lang/kernel/wave/utils/run_utils.py:97: RuntimeError
```

The error doesn't occur when the cache is turned on and when no cache exists.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.