alibaba / alibaba/clusterdata

Question about NaN GPU counts and End Times

Open
#198 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.2k
Forks
482
PR merge metrics
No merged PRs in 30d

Description

**Question 1:**

In `pai_task_table`, I have noticed that 223965 of the 1261050 rows have NaN `plan_gpu` values.

Is there a reason for these values being marked as NaN? Is there any additional information that can be obtained from this?

It seems in `cluster-trace-gpu-v2020/analysis/analysis.ipynb` that these rows are dropped from analysis.

**Question 2:**

I noticed that 299948 *other* rows have NaN `end_time` values. This set of rows includes failed tasks.

For those failed tasks, what does a NaN `end_time` mean?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading cluster-trace-gpu-v2020/analysis/analysis.ipynb and inspecting the referenced pai_task_table fields and rows with NaN plan_gpu or end_time. Done means determining and documenting what each NaN value represents, including the failed-task cases, and explaining whether dropping those rows is appropriate for analysis.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook
Domain
data
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.