alibaba / alibaba/clusterdata

Finding extra datapoints not present in the traces

Open
#134 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.2k
Forks
482
PR merge metrics
No merged PRs in 30d

Description

I am wondering if there is any way to get the following data (I did not find these readily available in the 2021 traces):
* All the utilization metrics (CPU and memory) at the node level and container level are relative values (percentages of total capacity). Is there a way to get the absolute values of these metrics (eg: the actual amount of CPU used vs allotted value)?
* Can we find the data on scheduling decisions made? (eg: resources requested by a microservice vs resources allotted)
* Can we find the actual capacities of various machines on the cluster? (eg: CPU size and memory capacity of individual machines)
* Can we find similar traces for offline jobs on these machines in the same timeframe?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the 2021 traces and any repository documentation that defines their available fields. Check each requested category—absolute utilization, scheduling decisions, machine capacities, and offline jobs—and document where the data is present or explicitly unavailable. Done means providing a source or clear answer for every bullet.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering, distributed-systems
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.