alibaba / alibaba/ROCK

Scheduler worker tasks and remote sandbox requests can block indefinitely on unresponsive HTTP calls

Open
#1,236 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
485
Forks
81
Avg merge
16h 12m
Merged PRs (30d)
8

Description

### Problem

Two HTTP call sites lack timeouts and can hang indefinitely when a worker or remote sandbox server becomes unresponsive:

1. `BaseTask.run_on_worker` in `rock/admin/scheduler/task_base.py` is awaited without any timeout inside the scheduler's bounded-concurrency runner. A single stuck worker holds its semaphore slot forever, starving the scheduler.
2. `RemoteSandboxRuntime._request` in `rock/sandbox/remote_sandbox.py` issues `requests.post` without a `timeout`, so a hung server blocks the calling thread indefinitely.

### Impact

Stuck workers/servers can wedge the admin scheduler and SDK callers, with no recovery until the process is restarted.

### Expected

Both call sites should enforce a timeout and surface the failure instead of blocking forever.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading BaseTask.run_on_worker in rock/admin/scheduler/task_base.py and RemoteSandboxRuntime._request in rock/sandbox/remote_sandbox.py, then inspect how their callers handle failures. Done means both HTTP-related operations enforce a timeout and surface failure instead of blocking indefinitely, while preserving the scheduler's bounded-concurrency behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.