alibaba / alibaba/ROCK

Scale sandbox proxy read throughput: multi-worker uvicorn + shared httpx pools

Open
#1,176 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
485
Forks
81
Avg merge
16h 12m
Merged PRs (30d)
8

Description

## What
The sandbox proxy / admin read path is bottlenecked on a single-process event loop, a per-request `httpx.AsyncClient`, a small Ray thread-pool executor, and a fixed DB pool. This raises read throughput by:

- Running the proxy role across multiple uvicorn workers (opt-in via `--workers` / `ROCK_PROXY_WORKERS`), while keeping the admin role and `local`/`test`/`dev` single-process (they own scheduler/Ray singletons and per-process in-memory/fakeredis state).
- Reusing shared httpx connection pools (separate control-plane and data-plane clients) instead of constructing a new `AsyncClient` on every request.

## Why
Under concurrent read load, per-request client construction and the small executor/pool sizes cap effective throughput and waste connections. Multi-worker + pooled clients spread CPU across cores and reuse keep-alive connections, removing the single-loop ceiling without regressing per-pod connection budgets.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the sandbox proxy startup entry point and trace the proxy role's uvicorn configuration, including --workers and ROCK_PROXY_WORKERS. Then trace the proxy request paths and current httpx client construction. Done means opt-in proxy workers and shared control-plane/data-plane clients work without changing admin or local/test/dev single-process behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.