Document the noms LOCK contract (<data_dir>/<db>/.dolt/noms/LOCK): acquire timeout, lifetime/staleness, external-waiter flock safety
- Dominant language
- Go
- Stars
- 24.4k
- Forks
- 873
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 120
Description
**Background — an orchestrator's need.** We manage the `dolt sql-server` lifecycle (start/stop/restart) from a supervisor that also coordinates other processes against the same data dir. To restart safely we need to know when the previous server has actually released its hold on a database and when a new server can take it — without guessing. Today we wait on the noms `LOCK` file with an arbitrary timeout (60s) because the lock's contract isn't documented anywhere.
**What's unclear today (and what the code actually does).** From reading `go/store/nbs`:
- `LOCK` is an **advisory `fslock`** at `//.dolt/noms/LOCK` (`journal.go:576`; `lockFileName`, `file_manifest.go:45`), held from DB open (`newJournalManifest`) until `Close()` — i.e. for the **entire server lifetime**, not a short critical section.
- On acquire, dolt waits only **`lockFileTimeout = 100ms`** (`file_manifest.go:46`); on timeout it either errors with `ErrDatabaseLocked` or **silently falls back to read-only**. This appears to be the root of the recurring "database is read only" / "locked by another process" reports (#10852, #11036, #10858).
- It's unspecified whether the lock survives a hard crash as a stale lock, how an operator clears it, and whether an external process may itself `flock()` the file to coordinate.
**Proposal.** Document the LOCK contract in the dolt docs (and/or a `noms/LOCK` README), covering: (a) it is an advisory flock held for the server's lifetime — there is no "max hold time," the correct wait is "until the holding process exits / closes"; (b) dolt's own 100ms acquire timeout and the read-only fallback, so operators understand why a too-fast restart yields a read-only or "locked" server; (c) crash/staleness behavior and the safe way to clear a stale lock; (d) explicitly, whether an external supervisor may `flock()` these files to coordinate hand-off, or whether that risks corrupting dolt's own locking.
**Design options (≥2):**
1. **Docs-only.** A "Storage locking" page in dolthub/docs describing the above; zero code change. Resolves the #10852/#11036 confusion at the source.
2. **Docs + a clearer error.** Same docs, plus replace the silent read-only fallback / generic "locked by another process" with a message that names the LOCK path and points at the doc (overlaps #10858's ask). Lets operators self-diagnose.
3. **Docs + a supported readiness/lock-state query.** Expose lock-held state (e.g. via the readiness signal in the companion request, or a `dolt` subcommand) so supervisors coordinate on a supported signal instead of poking the LOCK file directly — removes the need for external flock entirely.
Filing as a documentation/clarity request; option 1 alone would unblock us. Thanks.
---
_Filed from a downstream orchestrator (Gas City) that manages the `dolt sql-server` lifecycle; verified against current `dolthub/dolt` source._
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the LOCK handling in go/store/nbs/journal.go:576 and file_manifest.go:45-46, then review the existing dolthub/docs structure for an appropriate storage-locking page. Document the lifetime-held advisory lock, 100ms acquire timeout and read-only fallback, crash/staleness behavior, safe clearing, and whether external flock coordination is supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100