bug: gator supervisor retries transient failures forever despite a documented cap
@purp is already working on this.
Since Sep 14, 2026.
- Dominant language
- Rust
- Stars
- 8.7k
- Forks
- 1.3k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 253
Description
User Story
As a maintainer supervising a gator agent, I want the supervisor to stop after repeated transient failures, so that a wedged or permanently-failing gator surfaces for human attention instead of burning sandbox time indefinitely.
Problem Statement
OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES (default 5) is documented as a retry cap, but it does not cap anything. In scripts/agents/runtime/supervisor.sh, retry_watch_cycle() uses the value only to choose between two nearly identical log messages via a modulo test; both branches fall through to sleep_with_heartbeat and return, and the watch loop continues unconditionally. There is no exit path on repeated transient failures.
The operator-facing documentation states the opposite. .agents/skills/launch-openshell-gator/SKILL.md:313 tells the operator to "Let supervisor retry unless repeated failures hit the configured cap," which describes behavior that does not exist.
Impact / Why This Matters
A gator hitting a persistent auth, transport, or infrastructure failure is classified as transient_failure and retries forever. Because the supervisor never exits, the sandbox stays alive and no terminal_failure sentinel is ever written, so the documented operator signal for "this needs a human" never fires.
The current workaround is for an operator to notice the condition by reading logs and delete the sandbox manually. That is insufficient because the sentinel table is the documented mechanism for deciding whether human action is needed, and it cannot report a state the supervisor cannot reach. Nothing else bounds the run: there is no timeout or cleanup path elsewhere in the gator lifecycle, so an unattended failure is only ever found by someone going to look.
Acceptance Criteria
- With
OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES=N, the supervisor stops retrying after N consecutive transient failures rather than continuing indefinitely. - On hitting the cap, the supervisor emits a
terminal_failuresentinel with a reason identifying the transient failure, matching the sentinel contract the skill documents. - Consecutive-failure counting resets after a successful cycle, so intermittent failures spread over a long run do not accumulate to the cap.
-
OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES=0retains its existing meaning of unlimited retries, and that meaning is documented. -
.agents/skills/launch-openshell-gator/SKILL.mddescribes the actual behavior, including what the operator should do when the cap is hit.
Reproduction Steps
- Launch a gator with a low cap:
OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES=2 ./scripts/agents/run.sh --agent gator ... - Induce a repeatable transient failure, for example by invalidating the
ghcredential the sandbox uses so watch cycles fail on transport/auth. - Watch the supervisor log.
Observed: openshell-agent: transient watch failure N (<reason>); retrying in Ns repeats without bound; the wording alternates at multiples of the cap but the loop never exits and no terminal_failure sentinel is written.
Expected: the supervisor stops after 2 consecutive transient failures and reports status=terminal_failure.
Environment
- OpenShell: repository scripts at
5b9daab93(not a released artifact) - OS: macOS 15 (Darwin 25.6.0); the supervisor is plain bash and the defect is host-independent
- Component:
scripts/agents/runtime/supervisor.sh, gator agent runtime
Logs
scripts/agents/runtime/supervisor.sh:21 MAX_TRANSIENT_FAILURES="${OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES:-5}"
scripts/agents/runtime/supervisor.sh:140 if [[ "$MAX_TRANSIENT_FAILURES" -gt 0 ]]; then
scripts/agents/runtime/supervisor.sh:141 if [[ $((transient_failures % MAX_TRANSIENT_FAILURES)) -eq 0 ]]; then
# both branches only echo, then sleep and return
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.