intelligent-machine-learning / intelligent-machine-learning/dlrover

The xpu_timer fails to generate normal timeline information

Open
#1,470 10 comments 0 reactions 1 assignee Claimed by @cos120 View on GitHub
investigating report xpu_timer
Dominant language
Python
Stars
1.7k
Forks
219
Avg merge
10h 11m
Merged PRs (30d)
11

Description

**Describe the bug**
Hi, after building the xpu_timer, the relevant lib could not be found, and needed to be packaged and installed to /usr/local/lib and /usr/local/bin. The generated matmul_dp8.bin after all this was done seemed to be problematic.

Using xpu_timer_gen_tracing_timeline also seems to produce svg that does not properly reflect timeline information. Are there more documentation or communication group to discuss these issues?

"Stack count is low (8). Did something go wrong?" is displayed after the calltrace is generated.

**To Reproduce**

Build according to the xpu_timer documentation

**Logs or Screenshots**

![Image](https://github.com/user-attachments/assets/f60708a8-e788-4ae5-9168-3d08354d7680)

![Image](https://github.com/user-attachments/assets/32121264-90d7-40af-94c5-55a2a53b3e21)

**Expected behavior**
generate normal timeline info

**APP Info :**
- xpu_timer: 0.4.0-rc
- Torch 2.4.0
- cuda 12.5
- NCCL 2.22.3

**HARDWARE Info :**
- Device: GPU A800

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.