[Feature] Add an authoritative runtime contract for task liveness and recovery

Summary

Jcode needs an authoritative runtime contract for task liveness and recovery. The runtime should distinguish normal silence from a stale-suspected task, preserve ownership and progress evidence, reconcile state before cancellation or retry, and prevent unsafe downstream execution.

The goal is not to restart every task that appears quiet. The goal is to move mechanical safety invariants out of agent-specific policy and enforce them consistently in the runtime, while leaving host and workflow policy in AGENTS.md.

Problem

Today, liveness and recovery behavior is split across background execution, watchdogs, provider timeouts, swarm state, and agent instructions. This creates ambiguity and safety risks:

• lack of output can be mistaken for process failure;
• observer silence can trigger an unsafe kill or retry;
• a retry can duplicate a non-idempotent side effect;
• cancellation can target a reused PID or a process that no longer belongs to the task;
• daemon reloads and observer failures can leave task ownership unclear;
• downstream tasks can start while an upstream task has an unknown result;
• provider-level retry settings do not express task-level recovery semantics.

The existing runtime has useful partial primitives, including background status, progress, checkpoints, ownership metadata, process-group cancellation, heartbeat/stale handling in some task paths, and dependency states. They should be unified behind one documented contract instead of being inferred from separate paths or agent instructions.

Goals

1. Add one durable execution record for every Jcode-managed task.
2. Make task ownership and process identity verifiable before destructive actions.
3. Distinguish quiet, stale_suspected, external_wait, blocked, and observer_unknown.
4. Preserve heartbeat, progress, checkpoint, transition reason, and bounded event history.
5. Reconcile persistent state with observed state before cancellation, retry, or reclaim.
6. Make retry depend on idempotency, checkpoint, ownership, dependencies, and retry budget.
7. Block downstream work after an unsafe or unknown upstream result.
8. Expose machine-readable status and events through supported public interfaces.
9. Add safe, documented configuration for thresholds and recovery limits.
10. Reduce only the mechanical part of AGENTS.md after the runtime contract is accepted and tested.

Proposed runtime contract

Durable execution record

A managed task record should contain, at minimum:

• task identity, parent/plan identity, and dependency IDs;
• owner instance, owner process identity, process-group identity, and assigned session or agent;
• start time, last heartbeat, last progress, and observer state;
• typed progress events and bounded messages;
• checkpoint ID, timestamp, and resumability metadata;
• current state, transition reason, and bounded event history;
• declared external-wait classification;
• retry class, retry count, retry budget, idempotency class, and recovery action;
• bounded, redacted diagnostics with evidence age.

The record must not contain credentials, tokens, complete environments, or unbounded raw command payloads.

State model

The state model should distinguish at least:

queued, running, quiet, stale_suspected, diagnosing, external_wait, blocked, cancelling, retrying, completed, failed, cancelled, and observer_unknown.

Required semantics:

• no output alone must never imply failed;
• quiet means that no progress was observed, not that the process is dead;
• stale_suspected requires bounded diagnosis;
• external_wait and blocked must not be treated as ordinary process failures;
• observer_unknown is not proof that the owner exited;
• cancellation and retry require ownership, reconciliation, and an explicit policy;
• downstream work must not start while a prerequisite is unsafe or unknown.

Ownership-safe cancellation

Before cancellation, the runtime must verify task identity, owner instance, process identity, process group, PID-reuse safety, and the applicable policy. Cancellation must be limited to the confirmed task-owned process group.

The recommended sequence is:

1. record the transition and reason;
2. stop new downstream actions;
3. attempt graceful cancellation when allowed;
4. after a bounded grace period, force-kill only the confirmed task-owned group;
5. record the observed result and final state.

Unknown ownership must lead to bounded diagnosis or escalation, never to a broad kill.

Heartbeat, progress, and checkpoint

Heartbeat, progress, and checkpoint are separate signals:

• heartbeat shows that the owner is alive;
• progress shows that work changed;
• checkpoint shows a safe recovery point.

A long-running task may have no progress for a period while remaining healthy if its heartbeat is fresh or it is in a declared external wait. Lack of heartbeat and lack of checkpoint must not be treated as evidence that retry is safe.

Reconciliation and recovery

Before cancellation, retry, or reclaim, compare the durable record with observed state:

• whether the process still exists;
• whether owner and process group still match;
• whether the checkpoint changed;
• whether the side effect may already have completed;
• whether an observer or daemon reload occurred;
• whether a replacement task already exists.

If reconciliation is inconclusive, keep the task in observer_unknown or blocked and escalate in a bounded way. Do not start an automatic retry.

Retry and idempotency

Use explicit task-level retry classes:

• safe_to_retry;
• retry_from_checkpoint;
• retry_requires_reconciliation;
• non_idempotent_no_automatic_retry;
• unknown_no_automatic_retry.

Unknown and non-idempotent tasks must default to no automatic retry. Retry must consume a bounded budget and record its reason, checkpoint, observed side effects, and previous attempts.

Dependency gating

The scheduler must keep downstream tasks blocked while an upstream task is stale_suspected without completed diagnosis, observer_unknown, blocked, cancelling, or has an unknown side-effect result. Unblocking requires a terminal result or a validated checkpoint contract.

Diagnostics

Diagnostics must be bounded by time, size, and attempts, redacted, tied to an evidence timestamp, and available in machine-readable status. They must not expose full argv, environments, secrets, or raw sensitive payloads.

Configuration

Add an official task-level policy, for example [liveness], or an equivalent schema:

[liveness]
enabled = true
heartbeat_interval_secs = 15
stale_after_secs = 60
cancel_grace_secs = 10
max_diagnostic_duration_secs = 30
max_diagnostic_attempts = 1
auto_retry_unknown = false
auto_retry_non_idempotent = false
block_downstream_on_unknown = true

The numbers above are illustrative. Defaults must be selected from representative workload evidence. Unknown keys and invalid ranges must fail validation, safe defaults must work without extra configuration, and provider-level retry must not bypass task-level safety gates.

Suggested implementation issues

1. Introduce a durable ExecutionRecord and liveness state machine.
2. Enforce ownership-safe cancellation and process-group boundaries.
3. Unify heartbeat, progress, checkpoint, and reload persistence.
4. Add reconciliation, retry classes, idempotency handling, and recovery policy.
5. Add dependency gating for uncertain states.
6. Add official liveness configuration and bounded diagnostics.
7. Expose public status/event APIs and an end-to-end acceptance suite.
8. Reduce AGENTS.md only after every removed runtime rule has a tested runtime equivalent.

Acceptance criteria

The end-to-end suite must exercise real public/integration paths for:

• a normal silent task;
• a slow task with heartbeat;
• a task with no heartbeat and no output;
• declared external wait;
• stale owner and PID reuse;
• observer failure;
• graceful cancellation;
• force cancellation limited to an owned process group;
• safe retry from checkpoint;
• non-idempotent failure and uncertain side effect;
• already-completed side effect;
• blocked downstream task;
• daemon reload with record recovery;
• concurrent cancel/retry race;
• valid and invalid liveness configuration.

Each scenario must assert public status, event history, ownership decision, downstream behavior, and absence of forbidden side effects, not only an exit code.

Non-goals

• global scanning of unrelated host processes;
• universal timeouts without task semantics;
• blind restart or broad kill;
• automatic retry for unknown or potentially non-idempotent side effects;
• moving .env, Docker, Git publication, AppSec, QA, authority, or human-approval policy into the runtime.

Migration

1. Agree on states, transitions, ownership semantics, idempotency classes, public status schema, and compatibility behavior.
2. Implement the durable record, signals, ownership checks, and event history.
3. Add reconciliation, safe cancellation, retry gates, and dependency blocking.
4. Add configuration validation and the complete acceptance matrix.
5. Map every mechanical rule in AGENTS.md to a runtime guarantee and test.
6. Remove only duplicated mechanical instructions. Keep host/workflow policy and a short reference to the runtime contract.

This should be tracked as one umbrella feature with the implementation issues above because the safety guarantees depend on a shared execution record and state machine.

Start by reviewing AGENTS.md and the existing background execution, watchdog, heartbeat/stale handling, swarm state, dependency states, and public integration paths. The issue is an umbrella design effort: done requires agreed state and ownership semantics, a shared execution record and state machine, validated configuration, and the complete acceptance matrix, so the suggested implementation issues should be split before coding.