electric-sql / electric-sql/pglite

[BUG]: QueryQueueManager deadlocks when handler disconnects mid-transaction

Open
#985 2 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
16k
Forks
442
Avg merge
20h 19m
Merged PRs (30d)
7

Description

## Description

`QueryQueueManager.processQueue()` in `pglite-socket` permanently deadlocks when a handler that started a transaction disconnects before another handler's query can be processed.

When `isInTransaction()` returns `true` and `lastHandlerId` points to a handler that no longer has queries in the queue (because it disconnected), `processQueue()` sets `query = null`, breaks out of the `while` loop, sets `processing = false`, and **never processes queued queries from other handlers again**.

## Impact

This makes `pglite-socket` unusable with any client that opens multiple connections where one connection may issue a `SET` or `BEGIN` before the other connection's queries are processed. Hasura (GraphQL engine) reliably triggers this on every startup because it opens 2+ connections and sets transaction isolation level on its first connection.

## Steps to Reproduce

```js
import { PGlite } from "@electric-sql/pglite";
import { PGLiteSocketServer } from "@electric-sql/pglite-socket";

const db = new PGlite();
await db.waitReady;

const server = new PGLiteSocketServer({
db,
host: "0.0.0.0",
port: 5432,
maxConnections: 10,
});
await server.start();

// Now connect Hasura (or any client that opens 2+ connections):
// docker run --rm hasura/graphql-engine:v2.47.0 \
// -e HASURA_GRAPHQL_DATABASE_URL=postgres://postgres@host.docker.internal:5432/postgres?sslmode=disable
//
// Hasura opens connection A, sends startup + SET commands (enters transaction state).
// Hasura opens connection B, sends a query.
// processQueue sees isInTransaction()=true, lastHandlerId=A.
// Looks for A's queries in the queue — finds none (A's startup is done).
// Sets query=null, breaks. Queue permanently stuck.
```

With debug logging enabled, the last log line before the deadlock is:
```
[QueryQueueManager] transaction started, but no query from the same handler id found in queue
```

## Root Cause

In `processQueue()` ([source](https://github.com/electric-sql/pglite/blob/main/packages/pglite-socket/src/index.ts)):

```typescript
if (this.db.isInTransaction() && this.lastHandlerId) {
const idx = this.queue.findIndex(q => q.handlerId === this.lastHandlerId);
if (idx === -1) {
// BUG: sets query=null, which breaks out of the while loop
// and permanently stops queue processing
query = null;
} else {
query = this.queue.splice(idx, 1)[0];
}
}
```

The `clearTransactionIfNeeded()` method exists and handles cleanup on disconnect, but there's a race: if `processQueue()` is already executing when the disconnect fires, `processQueue()` exits before `clearTransactionIfNeeded()` can fix the state.

## Proposed Fix

When no matching handler query is found, rollback the stale transaction and process the next available query instead of deadlocking:

```typescript
if (idx === -1) {
this.log("rolling back stale transaction from disconnected handler", this.lastHandlerId);
await this.db.exec("ROLLBACK");
this.lastHandlerId = null;
query = this.queue.shift(); // process any waiting query
}
```

This is safe because:
- The handler that started the transaction is gone — its queries were already cleared by `clearQueueForHandler()`
- The transaction has no useful work to preserve
- Rolling back returns PGlite to a clean state so other handlers can proceed

## Environment

- `@electric-sql/pglite`: 0.4.5
- `@electric-sql/pglite-socket`: 0.1.5
- Node.js: 22.14.0
- Triggering client: Hasura GraphQL Engine v2.47.0 (uses libpq, opens multiple pooled connections)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.