lablup / lablup/backend.ai

Device-level allocation for multi agent

Open
#7,575 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

## Motivation

- A client has requested support for the following scenario
- A single agent/node will have multiple variations of NVIDIA GPU (e.g. some A100 and some H200 in the same server rack)
- The client has requested support for allocating kernels to specific GPU hardware
- The current Agent architecture does not support this, nor will we support device pinning in the near future
- As an alternative to supporting the above feature, we could use multi agent to define multiple agents in the same node, where each agent owns specific kind of devices

## Objective

- Ensure individual agent in multi agent use case can take ownership over specific devices
- Use Device UUID? Device name?
- Find a way of grouping devices based on its model
- Add a new CLI command for auto-generating agent.toml config that automatically creates and allocates multi agents grouped by device type

## Expected Sub Issue

- (Fix) Ensure multi agent is able to take ownership over specific devices
- Add configuration option for pinning specific device IDs to agents
- Add support for pinning specific devices to agents
- Add CLI command for auto-generating agent.toml config that splits accelerator devices per individual multi agent
- Ensure the above command can also support generating multi agent per device type

JIRA Issue: BA-3546

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by tracing the current Agent architecture and multi-agent device ownership, then clarify whether device UUIDs or names should drive allocation and grouping. Done requires agreed support for device pinning, device-type grouping, and CLI generation of the corresponding agent.toml configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, cli, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.