[gpu-v2026] Is job_execution_summary only supported from around Day 109?
- Dominant language
- Jupyter Notebook
- Stars
- 2.2k
- Forks
- 482
- PR merge metrics
- No merged PRs in 30d
Description
Hi ASI Trace Team,
Thank you for releasing the Alibaba Cluster Trace GPU v2026 dataset.
We are using the trace for task-resolved workload modeling and trace-driven GPU cluster scheduling. During our audit, we observed a very clear temporal boundary when linking `asi_opensource_pod_hourly` with `asi_opensource_job_execution_summary`.
Using only pod-ID coverage, before applying any Duration threshold or matching score, we obtain:
- Day 4–108: only about 0.2685% of reconstructed pod execution episodes have a pod ID that appears in `job_execution_summary`.
- Day 109–129: the coverage becomes 100%.
- Among our reliably linked execution episodes, 99.7845% come from Day 109 onward.
- Most of the small number of matches before Day 109 are long-running episodes that continue across the Day-109 boundary.
This suggests that the temporal boundary exists in the released data itself, rather than being caused by our execution matching procedure.
We noticed that Issue #257 also asks about the missing submission timestamp in `job_execution_summary`, and Issue #258 reports that `state_public` is mostly `Unknown` during roughly the first 100 days. Our observation may be related to the same metadata-coverage boundary.
Could you please clarify:
1. Does `asi_opensource_job_execution_summary` intentionally cover only executions from approximately Day 109 onward?
2. If so, is there any execution-summary data for Day 0–108 that could potentially be released?
3. Is there any anonymized field, mapping table, or execution-instance ID that can associate a row in `job_execution_summary` with a relative `day` / `hour` in `pod_hourly`?
4. How exactly is `duration_hours` defined? In particular, does it represent wall-clock execution span, active allocated execution time, or another quantity?
5. Would it be possible to release the upstream script used to construct `job_execution_summary`, or relative start/end timing information?
We do not need calendar dates, user identities, raw pod names, or any business-sensitive metadata. Relative day/hour information or a stable anonymized execution ID would be sufficient for our research.
We would be happy to share our audit scripts and aggregate linkage statistics if useful.
Thank you very much for releasing this valuable dataset.
Contributor guide
No contributing guide indexed for this repository
Research direction
Review Issues #257 and #258 alongside the documented pod_hourly and job_execution_summary fields. Compare the reported Day 4–108 and Day 109–129 coverage, then determine whether the dataset documentation can answer the coverage, timing, linkage, and duration questions. Done means the release documentation or maintainer response clearly states the supported period and available associations.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100