intel / intel/torch-xpu-ops

Host may stuck when submit too many kernels when event recording

Open
#2,467 1 comment 0 reactions 1 assignee Claimed by @Stonepia View on GitHub
dependency component: driver
Dominant language
Python
Stars
113
Forks
128
Avg merge
5d 13h
Merged PRs (30d)
107

Description

### 🐛 Describe the bug

run this reproducer as an example. If remove event record, it will be fine. Note l0 event pool size is 256 by default

Image

```python
import torch
import time
import os
a = torch.randn(1024,1024,1024, device="xpu")
b = torch.randn(1, device="xpu")

@torch.compile
def my_kernel(a, b):
return a + b
a = my_kernel(a, b)
torch.xpu.synchronize()

repeat = 20000
s = [torch.xpu.Event(enable_timing=True) for _ in range(repeat)]
e = [torch.xpu.Event(enable_timing=True) for _ in range(repeat)]
t = [0 for _ in range(repeat)]
os.environ["PTI_ENABLE_COLLECTION"] = "1"
for i in range(repeat):
t1 = time.time()
s[i].record()
a = my_kernel(a, b)
e[i].record()
t2 = time.time()
t[i] = t2 - t1
torch.xpu.synchronize()
for i in range(repeat):
print(s[i].elapsed_time(e[i]), t[i])

```

Image

Image

### Versions

PVC, main branch, oneapi 2025.2

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.