NVIDIA / NVIDIA/OpenShell

feat: make gateway Postgres connection pool size configurable (hardcoded to 10)

Open
#2,561 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

state:triage-needed
Dominant language
Rust
Stars
8.7k
Forks
1.3k
Avg merge
2d 11h
Merged PRs (30d)
253

Description

Problem Statement

The gateway's Postgres pool is hardcoded to max_connections(10) (crates/openshell-server/src/persistence/postgres.rs), with no config or env path. It's a single pool shared by every DB-backed RPC, so 10 is a hard, client-side ceiling on gateway↔DB concurrency — independent of how large the database is.

In cluster mode with external Postgres (workload.kind=deployment + server.externalDbSecret), a fleet of sandbox supervisors polling GetSandboxConfig / GetInferenceBundle / provider-env easily exceeds 10 concurrent connections. Symptoms:

  • sqlx::pool::acquire slow-threshold warnings; multi-second acquire latency on top of every DB-backed RPC.
  • /readyz becomes latency-sensitive to pool pressure, since it also acquires from this pool (PostgresStore::ping()). (This surfaced as readiness flapping in our case, but the root cause there was the default readiness-probe timeout being too short to tolerate a slow acquire — operator-tunable — not the pool itself. Noted only because it shows readiness is coupled to pool saturation.)

Scaling the database up doesn't help: the pool never opens more than 10 connections, leaving the extra capacity idle.

Proposed Design

Make the pool options configurable via environment variables, each defaulting to today's value (no change for existing deployments). Every one is read at startup and drops straight into the Helm chart as a container env: value — no image rebuild or config-file edit — consistent with OPENSHELL_DB_URL:

  • OPENSHELL_DB_MAX_CONNECTIONS — the pool cap (default 10). The primary fix.
  • OPENSHELL_DB_ACQUIRE_SLOW_THRESHOLD_SECS — the sqlx slow-acquire warn threshold (currently the sqlx default of 2s), so operators can tune or quiet that log line independently of the pool size.
  • Optionally OPENSHELL_DB_MIN_CONNECTIONS and OPENSHELL_DB_ACQUIRE_TIMEOUT_SECS.

Thread these into the pool builder (Store::connectPostgresStore::connectPgPoolOptions). They may also be mirrored in gateway.toml [openshell.gateway.database] (as log_level is today), with the env vars taking precedence.

Operator note: total connections = pool_size × replica_count, which must stay under the server's max_connections.

Agent Investigation
  • Single shared pool: Store::connect (crates/openshell-server/src/lib.rs) builds one PostgresStore/PgPool at startup.
  • Hardcoded cap: PgPoolOptions::new().max_connections(10).connect(url) (persistence/postgres.rs) — connect takes only the URL, no pool knob.
  • Shared by all handlers: e.g. handle_get_sandbox_config (grpc/policy.rs) does several store reads per request, each from the same 10 slots.
Checklist
  • I've reviewed existing issues and the architecture docs
  • This is a design proposal, not a "please build this" request

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Store::connect in crates/openshell-server/src/lib.rs and follow it into PostgresStore::connect and PgPoolOptions in crates/openshell-server/src/persistence/postgres.rs. Review how OPENSHELL_DB_URL reaches the Helm chart and how gateway.toml database settings are handled. Done means the pool settings are configurable at startup with today's defaults preserved, environment precedence defined, and the total connection guidance documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgres, rust
Domain
backend, database
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.