lablup / lablup/backend.ai

Derive the start-service launch budget from agent configuration instead of hardcoded constants

Open
#13,683 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Goal

Replace the two hardcoded service-launch timeouts with a single operator-tunable value, so the agent and the kernel runner always derive their budgets from the same number instead of two constants that can silently drift apart.

## Background

BA-7258 restored the ordering by raising the agent's reply timeout to a constant that exceeds the kernel runner's launch timeout. That is the minimal, backportable fix, but it leaves two problems open:

- The budget cannot be tuned per deployment. Sites whose images bind slowly under CPU or IO contention have no knob.
- The two constants live in packages that cannot import each other (`ai.backend.kernel` runs inside the container), so nothing prevents a future edit to one of them from re-inverting the pair.

The proposed shape, from the BA-7258 report: add a key to the existing `[kernel-lifecycles]` section of `agent.toml`, send it in the start-service payload, and have the kernel runner honour it. Older runners ignore the extra key, so this stays backward compatible.

## Acceptance criteria

- The launch budget is configurable through `[kernel-lifecycles]` in `agent.toml` and is etcd-overridable like its neighbours.
- The agent and the kernel runner derive their budgets from that one value, with the agent's reply timeout strictly greater than the runner's launch timeout.
- A regression test asserts the ordering so the pair cannot silently invert.
- A start-service payload without the key keeps the pre-existing runner behaviour, so containers created before the upgrade are unaffected.
- The generated `configs/agent/sample.toml` documents the new key.

## Notes

- The kernel runner terminates the service process when its launch timeout expires, so lowering the runner's budget is a behaviour change, not just a tuning knob. Pick the default with that in mind.
- The kernel runner ships bind-mounted from the agent's own venv, so no image rebuild is needed, but the mount is fixed at container creation: the change only reaches newly created containers.
- The effective ceiling on the agent's side is the client's own request timeout, which is outside this repository.

## Related

- BA-7258 — the minimal fix this builds on (backported to 26.8 and 26.4).
- BA-7260 — start-service replies are not correlated with requests; touches `feed_start_service` in the same file.

## Story points

1

JIRA Issue: BA-7322

Contributor guide

Open the contributing guide

Research direction

Start by locating the agent's [kernel-lifecycles] configuration and the feed_start_service start-service payload, then trace the kernel runner's launch-timeout handling. Check configs/agent/sample.toml for the generated configuration. Add coverage for the shared budget ordering and verify that payloads without the key retain the existing runner behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, distributed-systems, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.