electric-sql / electric-sql/pglite
Feature request: lock the data directory so a second PGlite in another process fails fast instead of corrupting it
- Dominant language
- TypeScript
- Stars
- 16k
- Forks
- 442
- Avg merge
- 20h 19m
- Merged PRs (30d)
- 7
Description
**New feature request**Postgres refuses to start on a data directory another postmaster holds (postmaster.pid). PGlite on the Node filesystem VFS takes no such lock, so two processes opening the same `dataDir` both succeed, both write, and the directory ends up unreadable. The next open then dies with `RuntimeError: Aborted()` (the symptom in #884) or a checkpoint panic (#327), with nothing pointing at the cause.
The two openers in our case were ordinary: a dev server running with `--hot`, and a script that imported the same database module. A `rm -rf` of the data directory under a running server ends the same way.
**Describe the solution you'd like**
`new PGlite(dataDir)` on the Node filesystem writes a lock file in the directory holding its process id, and a second open throws when that process is alive, naming it. A stale file (process gone) is taken over. Released on close and on process exit. Same idea as postmaster.pid, scoped to what PGlite can know in one process.
**Describe alternatives you've considered**
- We wrote this lock in the application (pid file beside the directory, `process.kill(pid, 0)` for liveness). It works, but every PGlite user with more than one process hits the same corruption and rediscovers it from `Aborted()`.
- A documentation note. It does not stop the second open.
**Additional context**
The lock request in #85 is for the IndexedDB VFS. This one is the Node filesystem VFS, where the second opener is another OS process.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the Node filesystem VFS path used by new PGlite(dataDir), then trace the open, close, and process-exit lifecycle. Reproduce the two-process case and verify that a live lock identifies its process, a stale lock is taken over, and the lock is released without corrupting the data directory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, postgresql, typescript
- Domain
- database
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100