openai / openai/codex-action

Responses API proxy hangs when using ARC runners with k8s

Open Beginner friendly
#108 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
1.2k
Forks
170
PR merge metrics
No merged PRs in 30d

Description

Summary

The Start Responses API proxy step backgrounds codex-responses-api-proxy with ( ... ) & and does not redirect its stdout/stderr. On self-hosted runners that deliver steps into a sidecar container via a WebSocket exec — notably Actions Runner Controller (ARC) in kubernetes container mode — the proxy then holds the parent step's exec stream open after bash exits. The next step's hook hangs until the websocket heartbeat times out (~1m52s in our environment) and the runner reports Executing the custom container implementation failed.

GitHub-hosted runners are unaffected (each step is a direct subprocess of the runner agent, so a backgrounded child holding stdio doesn't hold any exec stream open).

Symptom (v1.8)

##[group]Run (
  printenv PROXY_API_KEY | env -u PROXY_API_KEY "${args[@]}"
) &
...
responses-api-proxy listening on 127.0.0.1:NNNNN
[ ~1m52s of silence — proxy is up, no further step output ]
##[error]Executing the custom container implementation failed. Please contact your self hosted runner administrator.

In the runner pod's worker log, the hook process for the next step starts but never writes its response file. The 1m52s gap matches ARC's WebSocket heartbeat defaults (pingPeriodMs=5000, pongDeadlineMs=ping*12+1000≈61s) plus close-timeout / setup overhead.

Root cause

action.yml (v1.8), Start Responses API proxy step:

(
  printenv PROXY_API_KEY | env -u PROXY_API_KEY "${args[@]}"
) &

The subshell inherits the parent step's stdout/stderr. When bash exits, the backgrounded proxy keeps those FDs open for the lifetime of the job. On ARC k8s mode those FDs are the WebSocket exec stream, so the hook stays blocked.

Proposed fix

Two-line change. Detaches stdio while preserving proxy output in a log file under $RUNNER_TEMP for diagnostics. No behaviour change on github-hosted.

 (
   printenv PROXY_API_KEY | env -u PROXY_API_KEY "${args[@]}"
-) &
+) </dev/null >>"${RUNNER_TEMP:-/tmp}/codex-responses-api-proxy.log" 2>&1 &
+disown

Happy to send the PR.

Workaround we're using today

Runtime patch in the calling workflow, before invoking the action. The manifest is at _actions/openai/codex-action/<ref>/action.yml on the runner pod's work volume (mounted at /__w/_actions/… inside the $job container in ARC k8s mode):

- name: Patch openai/codex-action proxy step (ARC stdio detach)
  run: |
    set -euo pipefail
    action_yml=/__w/_actions/openai/codex-action/v1.8/action.yml
    [ -f "$action_yml" ] || { echo "::error::$action_yml missing" >&2; exit 1; }
    grep -qF 'codex-responses-api-proxy.log' "$action_yml" && { echo "already patched"; exit 0; }
    awk '
      BEGIN { in_block = 0 }
      /printenv PROXY_API_KEY \| env -u PROXY_API_KEY/ { in_block = 1; print; next }
      in_block && /^        \) &$/ {
        print "        ) </dev/null >>\"${RUNNER_TEMP:-/tmp}/codex-responses-api-proxy.log\" 2>&1 &"
        print "        disown"
        in_block = 0; next
      }
      { print }
    ' "$action_yml" > "$action_yml.new"
    grep -qF 'codex-responses-api-proxy.log' "$action_yml.new" \
      || { rm -f "$action_yml.new"; echo "::error::upstream shape changed; refresh patch" >&2; exit 1; }
    mv "$action_yml.new" "$action_yml"

- name: Run Codex review
  uses: openai/codex-action@v1.8
  with:
    # ...

We'd much rather drop this step.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in action.yml at the “Start Responses API proxy” step and inspect how the background process inherits its standard streams. Verify the proposed detachment and log-file behavior with an ARC Kubernetes container-mode runner; done means the following step no longer hangs and proxy output remains available under RUNNER_TEMP.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, kubernetes, shell
Domain
ci-cd, devops, infrastructure
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.