dolthub / dolthub/dolt

Document the noms LOCK contract (<data_dir>/<db>/.dolt/noms/LOCK): acquire timeout, lifetime/staleness, external-waiter flock safety

Open
#11,262 3 comments 0 reactions 0 assignees View on GitHub
customer issue documentation enhancement
Dominant language
Go
Stars
24.4k
Forks
873
Avg merge
1d 8h
Merged PRs (30d)
120

Description

**Background — an orchestrator's need.** We manage the `dolt sql-server` lifecycle (start/stop/restart) from a supervisor that also coordinates other processes against the same data dir. To restart safely we need to know when the previous server has actually released its hold on a database and when a new server can take it — without guessing. Today we wait on the noms `LOCK` file with an arbitrary timeout (60s) because the lock's contract isn't documented anywhere.

**What's unclear today (and what the code actually does).** From reading `go/store/nbs`:
- `LOCK` is an **advisory `fslock`** at `//.dolt/noms/LOCK` (`journal.go:576`; `lockFileName`, `file_manifest.go:45`), held from DB open (`newJournalManifest`) until `Close()` — i.e. for the **entire server lifetime**, not a short critical section.
- On acquire, dolt waits only **`lockFileTimeout = 100ms`** (`file_manifest.go:46`); on timeout it either errors with `ErrDatabaseLocked` or **silently falls back to read-only**. This appears to be the root of the recurring "database is read only" / "locked by another process" reports (#10852, #11036, #10858).
- It's unspecified whether the lock survives a hard crash as a stale lock, how an operator clears it, and whether an external process may itself `flock()` the file to coordinate.

**Proposal.** Document the LOCK contract in the dolt docs (and/or a `noms/LOCK` README), covering: (a) it is an advisory flock held for the server's lifetime — there is no "max hold time," the correct wait is "until the holding process exits / closes"; (b) dolt's own 100ms acquire timeout and the read-only fallback, so operators understand why a too-fast restart yields a read-only or "locked" server; (c) crash/staleness behavior and the safe way to clear a stale lock; (d) explicitly, whether an external supervisor may `flock()` these files to coordinate hand-off, or whether that risks corrupting dolt's own locking.

**Design options (≥2):**
1. **Docs-only.** A "Storage locking" page in dolthub/docs describing the above; zero code change. Resolves the #10852/#11036 confusion at the source.
2. **Docs + a clearer error.** Same docs, plus replace the silent read-only fallback / generic "locked by another process" with a message that names the LOCK path and points at the doc (overlaps #10858's ask). Lets operators self-diagnose.
3. **Docs + a supported readiness/lock-state query.** Expose lock-held state (e.g. via the readiness signal in the companion request, or a `dolt` subcommand) so supervisors coordinate on a supported signal instead of poking the LOCK file directly — removes the need for external flock entirely.

Filing as a documentation/clarity request; option 1 alone would unblock us. Thanks.

---
_Filed from a downstream orchestrator (Gas City) that manages the `dolt sql-server` lifecycle; verified against current `dolthub/dolt` source._

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the LOCK handling in go/store/nbs/journal.go:576 and file_manifest.go:45-46, then review the existing dolthub/docs structure for an appropriate storage-locking page. Document the lifetime-held advisory lock, 100ms acquire timeout and read-only fallback, crash/staleness behavior, safe clearing, and whether external flock coordination is supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.