intel / intel/llvm

[CI][HIP] Tracker ticket for flaky HIP CI issues

Open
#17,464 1 comment 0 reactions 0 assignees View on GitHub
bug hip
Dominant language
LLVM
Stars
1.5k
Forks
854
Avg merge
3d 17h
Merged PRs (30d)
137

Description

### Describe the bug

This ticket tracks all of the other tickets and disabled tests related to the flaky CI issues on the AMD runner.

It is specifically focusing on the cases where one or multiple test is hanging in the same run as one or multiple test is failing with a memory access fault.

The issue has been worked around by limiting the AMD CI to run on a single thread, so it shouldn't happen anymore, but this ticket is to investigate the issue and help close the tickets and re-enable the tests once we have figured out the actual issue.

**Tests with memory access fault**
```
Experimental/launch_queries/max_work_group_size.cpp
FreeFunctionCommands/mem_advise.cpp
Regression/commandlist/gpu.cpp
WeakObject/weak_object_expired.cpp
Reduction/reduction_nd_ext_half.cpp
Reduction/reduction_big_data.cpp
Basic/built-ins/vec_relational.cpp
Basic/built-ins/vec_math.cpp
SubGroup/sub_group_as.cpp
HostInteropTask/host-task-dependency3.cpp
Reduction/reduction_big_data.cpp
SharedLib/use_with_dlopen_verify_cache.cpp
```

**Tests hanging**
```
WorkGroupMemory/basic_usage.cpp
syclcompat/memory/memory_management_test2_usmnone.cpp
SubGroup/reduce_spirv13.cpp
Adapters/retain_events.cpp
SubGroup/scan.cpp
```

**List of related tickets**
* https://github.com/intel/llvm/issues/17441
* https://github.com/intel/llvm/issues/17415
* https://github.com/intel/llvm/issues/17339
* https://github.com/intel/llvm/issues/17284
* https://github.com/intel/llvm/issues/17283
* https://github.com/intel/llvm/issues/17236
* https://github.com/intel/llvm/issues/17235
* https://github.com/intel/llvm/issues/17194

**PR disabling related tests**
* https://github.com/intel/llvm/pull/17417
* https://github.com/intel/llvm/pull/17237

**Workaround PR with `-j1`**
* https://github.com/intel/llvm/pull/17422

### To reproduce

### Environment

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.