ory / ory/polis

Store init retries forever on a failed connect and never rejects (two unexitable while(true) loops)

Open
#4,079 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
2.3k
Forks
233
PR merge metrics
No merged PRs in 30d

Description

Summary

@boxyhq/saml-jackson initialises its SQL store inside two while (true) loops that catch every
error, sleep 1s, and retry indefinitely. Neither ever rejects, so a caller awaiting controller
construction cannot distinguish "still connecting" from "will never connect".

On a serverless platform this turns a routine credential expiry into a request that hangs until the
function is killed by the platform timeout.

Where

dist/db/sql/sql.js (v26.2.0), in the store init:

while (true) {                                  // loop 1 — connect
  try { /* ...build DataSource... */ await this.dataSource.initialize(); break; }
  catch (err) { this.logger.error(`error connecting to engine: ...`); await dbutils.sleep(1000); continue; }
}
// ...
while (true) {                                  // loop 2 — schema / index namespace
  try { if (synchronize) { await this.indexNamespace(); } break; }
  catch (err) { this.logger.error(`error in index namespace execution ...`); await dbutils.sleep(1000); continue; }
}

Both loops are reachable independently:

  • Loop 1 — wrong password / unreachable host: initialize() throws every attempt.
  • Loop 2 — a role that can connect but lacks CREATE on the schema: initialize() succeeds and
    the synchronise step throws every attempt.

Observed behaviour

A Postgres role with a revoked password produced, per request:

error connecting to engine: sql, type: postgres db: error: ... credentials are incorrect

repeated at ~1s intervals until the platform terminated the invocation. Two requests generated 466
such log lines. The endpoint returned a platform 504 after 300s, having held ~2 GB for the duration.

Related: the connect timeout does not bound this

sql.js computes connectionTimeoutMillis: pgOpts.connect_timeout * 1000, and parsePGOptions
(dist/db/utils.js) returns only what is present in the URL query string, with no default. Absent
?connect_timeout=, that expression is undefined * 1000NaN → falsy → no connect timeout.

Worth noting even so: adding a connect timeout would only make the retry loop spin faster. The
unbounded behaviour is the retry, not the connect.

Suggested fix

Bound the retry — a maximum attempt count, a deadline, or an option to fail fast — and reject
when it is exhausted, so callers can surface a real error. A configurable
initRetries / initTimeoutMs with a sane default would be enough; the current behaviour is only
safe for a long-lived process that can afford to wait forever, and is unsafe for any request-scoped
or serverless caller.

Workaround in use

Racing controller construction against a 10s deadline on our side and mapping the timeout to a 503.
The losing promise is not cancellable, so the orphaned init keeps retrying until the instance is
reclaimed — the caller is released, but the work is not stopped. A library-side bound would be
strictly better.

Version: 26.2.0. Engine: sql / postgres. Happy to supply a minimal reproduction if useful.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with dist/db/sql/sql.js, tracing both init while loops and the error paths for DataSource.initialize and indexNamespace; then inspect dist/db/utils.js for parsePGOptions and connect_timeout handling. Define bounded retry or deadline behavior and verify that failed credentials and schema permission errors reject instead of continuing to log and sleep indefinitely.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, typescript
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.