lablup / lablup/backend.ai

Add devices and kernel_devices tables (row models + migration + JSONB backfill)

Open
#13,449 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

Create two tables replacing the opaque kernels.attached_devices JSONB over the course of the epic: a devices entity table (agent-owned device inventory) and a kernel_devices junction recording per-kernel attachments.

Schema - devices:
- id UUID primary key (surrogate; keeps the junction FK single-column and the standard Updater path open for model_name updates)
- agent_id VARCHAR(64) NOT NULL references agents(id) - plain FK without cascade, same as kernels.agent
- device_name VARCHAR(64) NOT NULL (plugin key: cuda, cpu, ...)
- device_id VARCHAR(128) NOT NULL (plugin device id within the agent)
- model_name VARCHAR(255) NOT NULL
- created_at / updated_at TIMESTAMPTZ NOT NULL DEFAULT now(); updated_at refreshed when an upsert changes model_name
- UNIQUE (agent_id, device_name, device_id)

Schema - kernel_devices:
- kernel_id UUID NOT NULL references kernels(id) ON DELETE CASCADE
- device_id UUID NOT NULL references devices(id) ON DELETE CASCADE
- data JSONB NOT NULL DEFAULT empty object: the per-attachment allocation payload, stored losslessly as SSOT (plugin-owned open mapping; typed projection happens in memory via the entry models). Kept on the junction because allocation differs per kernel on the same device (fractional GPU)
- created_at TIMESTAMPTZ NOT NULL DEFAULT now()
- PRIMARY KEY (kernel_id, device_id); index on device_id for the device-to-kernels direction

The migration includes an idempotent two-step backfill from existing kernels.attached_devices (joined with kernels.agent for the device scope): upsert distinct devices, then insert junction rows with data, both conflict-safe, skipping elements without a device id and kernels without an agent.

Success Criteria
- [ ] alembic upgrade creates both tables and backfills from existing kernels; downgrade drops them
- [ ] backfill is idempotent: re-running produces no duplicates or errors
- [ ] deleting a kernel cascades to its kernel_devices rows; devices rows remain
- [ ] inserting a duplicate (agent_id, device_name, device_id) device violates the unique constraint
- [ ] pants test passes for affected packages

JIRA Issue: BA-7178

Contributor guide

Open the contributing guide

Research direction

Start with the Alembic migration and the existing kernels.attached_devices data joined through kernels.agent. Implement and verify the devices and kernel_devices schema, including conflict-safe backfill and cascade behavior, then run pants test for the affected packages. Done means alembic upgrade and downgrade work, rerunning the backfill is safe, and the listed constraints hold.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.