electric-sql / electric-sql/pglite

Feature request: lock the data directory so a second PGlite in another process fails fast instead of corrupting it

Open
#1,106 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
16k
Forks
442
Avg merge
20h 19m
Merged PRs (30d)
7

Description

**New feature request**Postgres refuses to start on a data directory another postmaster holds (postmaster.pid). PGlite on the Node filesystem VFS takes no such lock, so two processes opening the same `dataDir` both succeed, both write, and the directory ends up unreadable. The next open then dies with `RuntimeError: Aborted()` (the symptom in #884) or a checkpoint panic (#327), with nothing pointing at the cause.

The two openers in our case were ordinary: a dev server running with `--hot`, and a script that imported the same database module. A `rm -rf` of the data directory under a running server ends the same way.

**Describe the solution you'd like**

`new PGlite(dataDir)` on the Node filesystem writes a lock file in the directory holding its process id, and a second open throws when that process is alive, naming it. A stale file (process gone) is taken over. Released on close and on process exit. Same idea as postmaster.pid, scoped to what PGlite can know in one process.

**Describe alternatives you've considered**

- We wrote this lock in the application (pid file beside the directory, `process.kill(pid, 0)` for liveness). It works, but every PGlite user with more than one process hits the same corruption and rediscovers it from `Aborted()`.
- A documentation note. It does not stop the second open.

**Additional context**

The lock request in #85 is for the IndexedDB VFS. This one is the Node filesystem VFS, where the second opener is another OS process.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the Node filesystem VFS path used by new PGlite(dataDir), then trace the open, close, and process-exit lifecycle. Reproduce the two-process case and verify that a live lock identifies its process, a stale lock is taken over, and the lock is released without corrupting the data directory.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, postgresql, typescript
Domain
database
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.