lablup / lablup/backend.ai

Refactor and refine the resource usage API implementation

Open
#2,101 0 comments 0 reactions 1 assignee Claimed by @fregataa View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

We have a new resource usage summarization API to get the usage by project/session/kernel for given periods in #962.

However, we need to refactor the implementation.

- Avoid including unnecessary information in some outputs. For example, the per-project resource usage currently includes metadata about a session and kernel. Why??\* [In the code](https://github.com/lablup/backend.ai/blob/e77c4a3cd/src/ai/backend/manager/models/resource_usage.py), all `ProjectResourceUsage`, `SessionResourceUsage`, and `KernelResourceUsage` inherits `BaseResourceUsageGroup`. (Here, the name "Group" is _not_ related to the concept of projects.)
- `BaseResourceUsageGroup` contains direct references to a project, a session, and a kernel object. All subclasses merge the result of `to_json()` method of the superclass, and this incurs unnecessary inclusion of garbage information.
- Let's rewrite `BaseResourceUsageGroup` as a simpler **abstract** base class like `ResourceUsageAggregator` with just a few methods (no attributes) like `get_total_usage()`.
- Just refer this abstract base class when calculating the total usage.

- General tips\* When defining base classes and subclasses to share the type while differentiating the concrete implementations, do NOT hesitate to make empty classes or declaring just a few abstract methods. It's totally fine. Avoid putting too much subclass-specifics in the base classes.

- Rename/expand some fields.\* `ResourceUsage.nfs`: Currently it contains the list of host directory paths of mounted vfolders in the given resource set. "nfs" is a too implementation-specific name. We should relax the concept, like `ResourceUsage.vfolder_host_paths`.\* I think we need to add `ResourceUsage.volumes` to include the set of vfolder hosts (storage volumes) used by the given resource set.

JIRA Issue: BA-126

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.