Resource usage for task can be higher than resources requested
- Dominant language
- Jupyter Notebook
- Stars
- 2.2k
- Forks
- 482
- PR merge metrics
- No merged PRs in 30d
Description
There are many tasks in the dataset that utilize more resources than what was requested.
For instance, job_id:10771 task_id:66551 has plan_cpu:0.75 [1] from the following entry in _batch_task.csv_:
> 6301,6352,10771,66551,137,Terminated,**75**,0.01600704061294748
However, this task utilizes 7.66 (Max) and 0.99 (average) CPU as can be seen in batch_instance.csv:
> 6302,6339,10771,66551,427,Terminated,1,1,**7.66,0.99**,0.019309916392721248,0.012926772448424922
Can you clarify if the amount of resources used by tasks can be higher than the amount of resources requested? If not, what can explain these numbers?
Can I interpret the amount of resources requested as resources allocated by the scheduler?
[1] I divided the _plan_cpu_ value by 100 as explained in this issue: https://github.com/alibaba/clusterdata/issues/11
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the referenced entries in _batch_task.csv_ and batch_instance.csv, then review issue #11 for the plan_cpu interpretation. Compare the requested and observed CPU fields and trace any available dataset documentation or notebooks that define them. Done means documenting whether requested resources represent scheduler allocation and explaining the discrepancy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook
- Domain
- data
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100