nuclia / nuclia/hyperforge

SandboxSession kills workers when controller callbacks take >1s (WORKER_CPU_LIMIT hardcoded)

Open
#120 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3
Forks
1
Avg merge
1d 23h
Merged PRs (30d)
20

Description

Component

hyperforge.codemode.sandbox (SandboxRunner remote protocol / SandboxSession)

Summary

hyperforge/codemode/sandbox.py hardcodes WORKER_CPU_LIMIT = 1 (line 28). SandboxSession applies it as the wait budget while the controller answers a nested task request, not just while the worker is CPU-bound:

  • SandboxSession.run (line 306): await self._start_timeout(WORKER_CPU_LIMIT) — armed while waiting on the worker task.
  • SandboxSession._callback (line 361): await self._start_timeout(WORKER_CPU_LIMIT) — armed while waiting for the controller to respond to a SandboxMessage.Request (a task issued by generated code, answered by the SandboxRunner callback).

When the wait exceeds 1 second, _timeout fires and kills the worker process:

async def _timeout(self, timeout: float):
    await asyncio.sleep(timeout)
    self.timed_out = True
    if self.process.is_alive():
        self.process.kill()
Impact

Any legitimate nested capability dispatched through the controller callback that takes more than 1 second (a database query, an HTTP call, a paginated lookup) causes the entire sandbox worker to be killed with RuntimeError("Python agent timeout"). Embedding applications have no way to raise this bound; it is not part of SandboxSettings and not a parameter of run_sandbox_server().

Verified on hyperforge==1.0.0.post310.

Proposal

Move the value into SandboxSettings with the current value as the default (fully backward compatible), e.g.:

class SandboxSettings(BaseSettings):
    ...
    sandbox_callback_wait_seconds: float = 1.0  # replaces WORKER_CPU_LIMIT usage

Worker-runtime enforcement via WorkerExecutionRequest.max_runtime_seconds is separate and should be unaffected.

Acceptance criteria
  • The controller-callback wait is configurable via settings/env, default unchanged (1.0s).
  • max_runtime_seconds enforcement behaves exactly as before.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in hyperforge/codemode/sandbox.py at SandboxSession.run and _callback, then trace how SandboxSettings is exposed through run_sandbox_server. Confirm the callback wait uses a configurable setting while WorkerExecutionRequest.max_runtime_seconds remains unchanged. Done means the default remains 1.0 seconds, settings or environment can raise the callback wait, and the existing timeout behavior is preserved otherwise.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.