Nothing bounds how long a run can take
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 5
- Forks
- 10
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 17
Description
Raising the socket idle timeout to 255s and adding a keepalive (#629, #630) means a stream no longer goes idle, so nothing on the Apollo side bounds how long a run can take. The only timers in platform/src now are the 5s SIGKILL escalation and the socket idle timer the keepalive deliberately defeats.
A Python service that wedges — an HTTP call with no timeout, a retry loop that never gives up — holds a socket, an interval and an interpreter for as long as the caller stays attached, and the keepalive is what keeps the caller attached. N concurrent requests to a wedged service pin N interpreters with no ceiling.
Cancellation covers the case where the caller leaves: the abort fires, the child gets SIGTERM. The gap is the caller that waits.
In practice Lightning bounds this from its side with APOLLO_REQUEST_TIMEOUT_MS, so today it is not unbounded — but that is Lightning defending itself, not Apollo. Any other client, or a change to that setting, and the ceiling is gone.
What it probably wants is a run deadline in bridge.ts: an AbortSignal.timeout(n) composed with the caller's signal, firing the same path cancellation already uses, with its own error type so it reads differently from a client disconnect.
The open question is the number. It has to sit above the slowest legitimate run — p99 of workflow generation is around 50-57s — and below whatever the callers use, or Apollo never gets to be the one that explains what happened.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in platform/src/bridge.ts and trace the existing caller-cancellation path, then review how the socket, interval, and interpreter are managed during a run. Define the run deadline and distinct timeout error with the maintainer, accounting for the stated p99 workflow duration. Done means a wedged run is bounded without changing client-disconnect behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, typescript
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100