randomparity / randomparity/kdive
An empty-ControlGroup unit outside two narrow shapes is refused, blocking status, stop and recover alike
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 1h 26m
- Merged PRs (30d)
- 311
Description
## Problem
A worker unit left with an **empty `ControlGroup`** in any state except two narrow ones is refused
by unit observation before any subcommand's own logic runs. `status`, `stop` and `recover` all fail
the same way, so the slot cannot be inspected, stopped or recovered — the shape every one of those
subcommands exists to resolve.
The refusal message compounds it: it says the unit identity is *partial*, when the identity is
complete and it is the **state combination** that was not anticipated. An operator reading it looks
for a truncated `systemctl show`, which is not what happened.
## Evidence
`src/kdive/processes/lifecycle/systemd/systemd_worker_runtime.py:574-581` — when `ControlGroup` is
empty, observation delegates to a terminal-identity check rather than accepting the unit:
```python
if control_group:
...
membership = self._membership(control_group)
else:
self._require_released_terminal_identity(properties)
membership = "empty"
```
`:864-877` — that check accepts exactly two shapes and raises otherwise:
```python
failed = (
properties["ActiveState"] == "failed"
and properties["SubState"] == "failed"
and properties["Result"] != "success"
)
remain_after_exit = (
properties["ActiveState"] == "active"
and properties["SubState"] == "exited"
and properties["Result"] == "success"
and properties["ExecMainStatus"] == "0"
)
if not failed and not remain_after_exit:
raise SystemdConflict("systemctl show returned a partial unit identity")
```
So an `active (exited)` unit whose `Result` is not `success`, or whose `ExecMainStatus` is not `0`,
matches neither arm. Both are reachable: `RemainAfterExit` keeps a unit `active (exited)` after its
main process exits, and the exit status it exits with is not constrained to `0`.
`SystemdConflict` raised during observation is not caught by a per-subcommand recovery path, so it
surfaces identically for `status`, `stop` and `recover`.
## What is established, and what is not
**Established by reading the source:** the two accepted shapes above, that anything else raises, and
that the message names a partial identity rather than an unexpected state combination.
**Observed, not reproduced here:** the #2533 implementer hit this on a provisioned Fedora 44 host
while running that issue's acceptance arms. A failed `start 2` left slot 1 `active (exited)` with an
empty `ControlGroup`, and `status`, `stop` and `recover` all refused before any of that branch's
code ran. The exact `Result` and `ExecMainStatus` values on that unit were not captured, so which of
the two disqualifying conditions applied is not known. The host has since been returned to a clean
state, so the unit is gone.
Whoever takes this should capture `systemctl show` for the affected unit first; the fix depends on
which combination occurs in practice, and guessing it from the source is how a guard ends up
accepting the wrong set.
## Expected
A retained unit with an empty `ControlGroup` and no live process is observable, so `status` can
report it and `stop` or `recover` can act on it — or, if some combination genuinely must be refused,
the refusal names the state it saw rather than describing the identity as partial.
## Proposed approach
1. Capture the real property set from an induced failure before changing the condition. A failed
`start` against a second slot is how it was reached.
2. Decide whether `active (exited)` with a non-zero `ExecMainStatus` is a legitimate terminal
observation. An empty control group means no process is running, which is the fact the callers
need; the exit status is evidence about how it ended, not about whether it is observable.
3. If a combination must still be refused, make the message carry `ActiveState`, `SubState`,
`Result` and `ExecMainStatus` so the operator sees which one disqualified it.
4. Add a regression per accepted and refused combination. This gap exists because the two accepted
shapes were enumerated positively with no test for what falls between them.
## Out of scope
- The five residual cases #2533 covers. This is a sixth shape that issue does not name, and its
acceptance does not reach it.
- Whether `recover` should clear a slot it cannot observe. That is the same prohibition ADR-0657
places on unreadable identity and should not be relaxed as a side effect.
## Provenance
Found by the implementer of #2533 while running that issue's seventh acceptance criterion on a real
provisioned host. Reported as outside its five cases and in ADR-0574 / #2532 territory, and
deliberately not fixed there. The source mechanism was verified independently before filing; the
occurrence is reported rather than reproduced, and this issue says so rather than implying a
reproduction it does not have. Filed at the repository operator's explicit request.
Contributor guide
Research direction
Start in src/kdive/processes/lifecycle/systemd/systemd_worker_runtime.py:574-581 and 864-877, then capture the affected unit's systemctl show output by inducing the reported failed start before changing the condition. Check the observation paths for status, stop and recover. Done means the real empty-ControlGroup state is handled or rejected with a state-specific message, with regression coverage for each accepted and refused combination.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, python
- Domain
- backend, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100