Worker Framework Timeout Leak - Zombie Workers Persist Across Restarts

GitHub Issue: Worker Framework Timeout Leak - Zombie Workers Persist Across Restarts

Summary

Worker spawn failures not caught by timeout mechanism, creating zombie workers that persist across container restarts.

Symptoms

• Workers spawn but execute 0 tool calls
• Timeout (500s) never triggers on spawn failures
• Workers persist in "running" state for thousands of minutes
• Container restarts don't clear zombie workers
• Worker queue becomes clogged, blocking new executions

Evidence

• 11 zombie workers currently active (run times: 645-1070 minutes)
• Multiple container restarts failed to clear them
• All affected workers show "0 tool calls executed"
• Timeout only triggers for workers that successfully start, not spawn failures

Root Cause

The worker timeout mechanism only monitors workers after successful spawn/initialization. If a worker fails to spawn (e.g., due to provider configuration errors), it never reaches the execution phase, so the timeout never starts.

Expected Behavior

Workers that fail to make their first tool call within 60 seconds should auto-terminate, regardless of spawn success.

Actual Behavior

Workers that fail during spawn remain in "running" state indefinitely, consuming resources but executing nothing.

Impact

• Resource leaks (CPU/memory consumed by dead workers)
• Worker queue clogging (new workers can't spawn)
• Entire agent execution pipeline down
• Cannot automate configuration or monitoring tasks

Workaround Used

Completed phase 2.1 implementation via direct shell execution (bypassing workers entirely).

Proposed Fix

1. Add spawn-failure timeout (60s) - kill workers that never execute
2. Add max-spawn-duration config parameter
3. Ensure container restart properly terminates all worker processes
4. Add health check: auto-terminate workers with 0 tool calls after 5 minutes

Severity

Critical - Blocks all automated agent execution

Start by locating the worker spawn, initialization, timeout, and restart-cleanup paths in the Rust worker framework. Reproduce a spawn failure and inspect how workers with zero tool calls are recorded and terminated. Done means failed or inactive workers are cleaned up within the specified timeout and the queue can accept new executions.