Migrate kernel attached devices from JSONB to a dedicated table with typed entry models
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
Kernel attached-device info is stored as an opaque JSONB column (kernels.attached_devices) written once at the RUNNING transition (update_kernel_status_running in repositories/scheduler/db_source/db_source.py) and consumed only by usage/period aggregation (models/resource_usage.py and repositories/group/db_source/db_source.py, reading model_name plus data.smp / data.mem per cuda device). It is not exposed in any API today.
This epic normalizes the storage into a devices entity table plus a kernel_devices junction, introduces typed entry models, and exposes the data in the v2 API.
Design decisions:
- Tables: devices is an agent-owned device inventory (surrogate id, UNIQUE (agent_id, device_name, device_id), plain agents FK like kernels.agent, model_name updatable via upsert) and kernel_devices is the attachment junction (composite PK (kernel_id, device_id), kernels FK CASCADE, index on device_id for the device-to-kernels direction).
- The per-attachment data payload stays on the junction as a raw JSONB map (lossless SSOT): it is the allocation for that kernel, which differs per kernel on the same device under fractional GPU scaling, and it is a plugin-owned open mapping whose actual keys (smp, mem, cores) already drift from the declared ComputedDeviceCapacity type, so it cannot be safely columnized. Device identity fields (device_name, device_id, model_name) live on the entity table.
- Typed entry models (AttachedDeviceEntry with nested DeviceCapacityEntry name/value pairs) follow the ResourceSlotEntry precedent: keys stay open, values are constrained to numerics, projection happens in memory and at the API boundary.
- Write path: no new event handler; upsert devices and insert junction rows in the same transaction as the guarded RUNNING UPDATE (WHERE status IN PREPARED, CREATING), so the existing rowcount guard provides idempotency for free. Existing JSONB writes are kept as dual-write until the read switch.
- Exposure: KernelV2.attachedDevices connection field in GQL (authorization inherited from kernel node reachability) plus a kernel-scoped REST v2 endpoint authorized via ResolveKernelSessionAction (owning-session access, scheduling-history precedent).
Phases: entry models + tables (parallel) - write path dual-write - v2 exposure - read switch and JSONB column drop.
JIRA Issue: BA-7176
Contributor guide
Research direction
Start with update_kernel_status_running in repositories/scheduler/db_source/db_source.py, then trace attached-device reads in models/resource_usage.py and repositories/group/db_source/db_source.py. Review the ResourceSlotEntry precedent and the phased design for entry models, tables, dual-write, v2 exposure, read switch, and JSONB removal. Done means the migration and all listed write, read, and API phases are implemented without losing attachment data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- graphql, postgresql, python
- Domain
- api, backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100