lablup / lablup/backend.ai

Exclude `SYSTEM` sessions from resource allocation while keeping per-container resource limits

Open
#1,672 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

Currently, SYSTEM session containers also occupy the resource allocation and sometimes this confuses our users and admins when they figure out the remaining resources available for _computing_.

We should make it intuitive to see the remaining "usable" resources, and SYSTEM session containers makes it difficult as they are hidden but still affects the scheduling process. It is hard to design the UI to display the remaining "usable" resources as the actual agent occupation differs from what the user see in the UI's resource indicators. The problem becomes more complicated if we consider `hide_agents=false` configuration.

Let's resolve this situation by making SYSTEM session containers to have _only resource limits_ but not resource allocation:

- Assume that they _DO NOT use accelerators_ and require relatively small amount of CPU cores and memory, allowing oversubscription.
- Just keep track of _the maximum number_ of SYSTEM session containers per agent.
- Just use the minimum CPU/MEM resource requirements to set the resource limits.
> **Note**
>
> - _\*Resource allocation:\*_ Assign an exclusive, dedicated portion of resources which the scheduler will treat as "consumed". (manager/agent-level scheduling)
> - _\*Resource limit:\*_ Ensure the container cannot use more resources exceeding the configured limit. (container's cgroup configuration, jail, hook, and some env-vars)
> - Normal Backend.AI session containers have both.

This conceptual model change make it possible to merge #710 into the SFTP-only resource group and hidden agents running along with the storage proxy. Both SFTP sessions and filebrowser sessions could be treated in the same way.

**Expected scope:**

- We may need to add many branches in the scheduler to apply the different semantics to SYSTEM pending sessions. For instance, we should skip or override the normal scheduler and agent selection strategy for them.\* By default, we could apply a simple load balancing of SYSTEM sessions by selecting the agent with the minimum number of running SYSTEM sessions within the target resource group.

- After implementation, we could exclude SYSTEM sessions from the normal session listing APIs, except for when explicitly requested to include it via a new option.

JIRA Issue: BA-182

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.