Improve scheduling history visibility in manager
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
## Background
Scheduling failures are recorded into `status_data` and the `scheduling_history` table as a flattened string (`SchedulingFailure.msg = str(e)`). WebUI / CLI render this string verbatim. Today the operator experience has two concrete pain points:
1. \***Multi-node context is lost.**\* When a multi-node session fails mid-batch (e.g., 3rd of 4 kernels can't fit on any agent), the surfaced error reflects only the last failing iteration. Cluster size, the kernel index that failed, and which agents had already received earlier kernels in the same session are discarded at the exception boundary. Operators can't tell whether `available=0` means the agent is genuinely full or earlier kernels in the same session consumed the slot.
2. \***Single-line `;`-joined messages are unreadable.**\* Per-slot details (`InsufficientResourcesError.__str__`) and per-agent aggregations (`_select_agent_tracker_for_requirements`) both use `; ` as separator, so long failure messages run into a single dense line. WebUI/CLI cannot present these structurally.
## Scope
Two child Stories:
- Surface multi-node scheduling context in failure messages
- Switch failure-message separators from `;` to `\n` so WebUI/CLI can split per line
Both target the same files (`selectors/exceptions.py`, `selectors/selector.py`, `provisioner.py`) but in non-overlapping ways. Recommended landing order: separator change first, then the multi-node enrichment uses newline from the start.
WebUI / CLI side will adapt to the newline-delimited format once the manager change ships.
## Out of Scope
- WebUI / CLI rendering changes (tracked separately on those repos)
- Functional changes to scheduling decisions
JIRA Issue: BA-6147
Contributor guide
Assessment
This issue has not been assessed yet.