oneapi-src / oneapi-src/level-zero-tests

"ze_peak" freezes on DG1 with latest drm-tip kernel + drivers

Open
#20 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
64
Forks
69
PR merge metrics
No merged PRs in 30d

Description

Setup:

  • HW: CML-S / DG1 (0x4905)
  • OS: Ubuntu 22.04
  • Kernel: "drm-tip" head from yesterday
  • UMD: Latest releases of compute stack components, built with LLVM 12
  • App: "ze_peak" from level-zero-tests head

Bug:

./ze_peak freezes with 99% CPU usage after showing:
Single Precision Compute (GFLOPS)

(I.e. half precision and global BW tests before it worked fine.)

It can be quit with ^C, so it's not in 100% CPU loop.

Gdb shows:

warning: Target and debugger are in different PID namespaces; thread lists and other data are likely unreliable.  Connect to gdbserver inside the container.
0x00007f6fbca28cab in sched_yield () from target:/lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0  0x00007f6fbca28cab in sched_yield () from target:/lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f6fbc27cd63 in ?? () from target:/usr/local/lib/libze_intel_gpu.so.1
#2  0x00007f6fbc0572c2 in ?? () from target:/usr/local/lib/libze_intel_gpu.so.1
#3  0x0000564de2c87d3f in ?? ()
#4  0x0000564de2c88653 in ?? ()
#5  0x0000564de2c94ba1 in ?? ()
#6  0x0000564de2c86104 in ?? ()
#7  0x00007f6fbc949d90 in ?? () from target:/lib/x86_64-linux-gnu/libc.so.6
#8  0x00007f6fbc949e40 in __libc_start_main () from target:/lib/x86_64-linux-gnu/libc.so.6
#9  0x0000564de2c862e5 in ?? ()

perf showed most of the time being spent inside libze_intel_gpu.so.1. I.e. it could be driver issue, but I thought it better to start from the app.

ze_image_copy, ze_nano and ze_pingpong work fine. ze_bandwidth gets slower and slower, and I did not wait for it to complete.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the freeze with ze_peak on the stated CML-S/DG1 setup and the listed drm-tip and compute-stack versions. Compare its behavior with ze_image_copy, ze_nano, ze_pingpong, and ze_bandwidth, then use the provided gdb and perf observations to determine whether the issue is in the test or libze_intel_gpu.so.1. Done means the cause is isolated and a regression test or fix is identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.