Responses API proxy hangs when using ARC runners with k8s
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1.2k
- Forks
- 170
- PR merge metrics
- No merged PRs in 30d
Description
Summary
The Start Responses API proxy step backgrounds codex-responses-api-proxy with ( ... ) & and does not redirect its stdout/stderr. On self-hosted runners that deliver steps into a sidecar container via a WebSocket exec — notably Actions Runner Controller (ARC) in kubernetes container mode — the proxy then holds the parent step's exec stream open after bash exits. The next step's hook hangs until the websocket heartbeat times out (~1m52s in our environment) and the runner reports Executing the custom container implementation failed.
GitHub-hosted runners are unaffected (each step is a direct subprocess of the runner agent, so a backgrounded child holding stdio doesn't hold any exec stream open).
Symptom (v1.8)
##[group]Run (
printenv PROXY_API_KEY | env -u PROXY_API_KEY "${args[@]}"
) &
...
responses-api-proxy listening on 127.0.0.1:NNNNN
[ ~1m52s of silence — proxy is up, no further step output ]
##[error]Executing the custom container implementation failed. Please contact your self hosted runner administrator.
In the runner pod's worker log, the hook process for the next step starts but never writes its response file. The 1m52s gap matches ARC's WebSocket heartbeat defaults (pingPeriodMs=5000, pongDeadlineMs=ping*12+1000≈61s) plus close-timeout / setup overhead.
Root cause
action.yml (v1.8), Start Responses API proxy step:
(
printenv PROXY_API_KEY | env -u PROXY_API_KEY "${args[@]}"
) &
The subshell inherits the parent step's stdout/stderr. When bash exits, the backgrounded proxy keeps those FDs open for the lifetime of the job. On ARC k8s mode those FDs are the WebSocket exec stream, so the hook stays blocked.
Proposed fix
Two-line change. Detaches stdio while preserving proxy output in a log file under $RUNNER_TEMP for diagnostics. No behaviour change on github-hosted.
(
printenv PROXY_API_KEY | env -u PROXY_API_KEY "${args[@]}"
-) &
+) </dev/null >>"${RUNNER_TEMP:-/tmp}/codex-responses-api-proxy.log" 2>&1 &
+disown
Happy to send the PR.
Workaround we're using today
Runtime patch in the calling workflow, before invoking the action. The manifest is at _actions/openai/codex-action/<ref>/action.yml on the runner pod's work volume (mounted at /__w/_actions/… inside the $job container in ARC k8s mode):
- name: Patch openai/codex-action proxy step (ARC stdio detach)
run: |
set -euo pipefail
action_yml=/__w/_actions/openai/codex-action/v1.8/action.yml
[ -f "$action_yml" ] || { echo "::error::$action_yml missing" >&2; exit 1; }
grep -qF 'codex-responses-api-proxy.log' "$action_yml" && { echo "already patched"; exit 0; }
awk '
BEGIN { in_block = 0 }
/printenv PROXY_API_KEY \| env -u PROXY_API_KEY/ { in_block = 1; print; next }
in_block && /^ \) &$/ {
print " ) </dev/null >>\"${RUNNER_TEMP:-/tmp}/codex-responses-api-proxy.log\" 2>&1 &"
print " disown"
in_block = 0; next
}
{ print }
' "$action_yml" > "$action_yml.new"
grep -qF 'codex-responses-api-proxy.log' "$action_yml.new" \
|| { rm -f "$action_yml.new"; echo "::error::upstream shape changed; refresh patch" >&2; exit 1; }
mv "$action_yml.new" "$action_yml"
- name: Run Codex review
uses: openai/codex-action@v1.8
with:
# ...
We'd much rather drop this step.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in action.yml at the “Start Responses API proxy” step and inspect how the background process inherits its standard streams. Verify the proposed detachment and log-file behavior with an ARC Kubernetes container-mode runner; done means the following step no longer hangs and proxy output remains available under RUNNER_TEMP.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, kubernetes, shell
- Domain
- ci-cd, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100