dolthub / dolthub/doltgresql

[RFC] Dolt-native Matrix homeserver + gaps re-checked on Doltgres 1.3.1

Open
#3,283 0 comments 0 reactions 0 assignees View on GitHub
customer issue
Dominant language
Go
Stars
2.1k
Forks
73
Avg merge
1d 10h
Merged PRs (30d)
129

Description

Hey @timsehn @nicktobey, opening this per your suggestion on dolthub/doltlite#2563 — the Doltgres-side follow-up to my [August update in #2099](https://github.com/dolthub/doltgresql/issues/2099#issuecomment-5390017737) (that one landed after the issue closed, so linking it as background; everything here stands on its own).

TL;DR:

Same prototype, one participant richer — two unmodified matrix-commander clients plus **a browser** in the same room, all through the Doltgres homeserver:

_(GIF is on heavy side, so slow to load)_
![three participants, one room, one id space — watched in dolt.log](https://github.com/thrudhame/communico/blob/spike/communico-lite/demo/communico-demo-3p.gif?raw=true)

To play with it yourself:

```bash
git clone -b spike/communico-lite https://github.com/thrudhame/communico.git
cd communico
bash demo/setup.sh # builds container, starts Doltgres + server, creates a room
bash demo/run-demo.sh --check # headless self-test, exits 0
bash demo/run-demo.sh --scripted # the tmux choreography from the GIF (the browser joins unseen)
```

## Idea

Unchanged from August: rooms as databases, events as commits, room state = the database at a commit, extremities = branches, federation = `dolt pull`. Details in `demo/inspection-tour.md`.

## Friction

Last time I had two room modes (spec-compliant vs "native", where the wire event id WAS the commit hash). That's gone, the multi-replica work killed it: the same logical event gets a different commit hash on every store that ingests it independently, and a browser peer (DoltLite, 40-hex hashes) and Doltgres (base32) don't even share a hash universe. So now:

- **one core**: the content-hash event id (Matrix's own reference-hash shape) is canonical for every room; the commit hash is each store's private receipt, mapped in `event_index`;
- **protocols are hats on top**: the Matrix client API is one, the browser peer's sync protocol is another; the core knows neither. Matrix is the protocol, Dolt is the implementation.

Everything else Dolt still carries as-is: state via `AS OF`, healing via the conflict-driver merge, extremities as branches, and the convergence proof is `dolt_hashof_table` equality across replicas.

## Gaps

Re-checked every item from August on **Doltgres 1.3.1** before filing — two got fixed along the way (thanks!), six stand. Verbatim errors and the re-check table: [`spikes/RESULTS.md` : "Gap re-check on Doltgres 1.3.1"](https://github.com/thrudhame/communico/blob/spike/communico-lite/spikes/RESULTS.md#gap-re-check-on-doltgres-131-2026-09-04). Curious which are principled and which are just "nobody asked yet":

1. **N-parent commits.** Matrix allows up to 20 `prev_events`; Dolt merges are 2-parent, so I chain merges. `DOLT_MERGE('mb','mc')` → `Dolt does not support merging from multiple commits…`. Probably principled, is an octopus merge on the table at all? **Still reproduces.**
2. **Conflicted merges can't be staged in autocommit.** Without `SET dolt_allow_commit_conflicts = 1`, `DOLT_MERGE` on a conflict rolls back and no conflict state survives the statement — nothing to inspect or resolve. The session var unblocks my driver pattern (the whole room engine depends on it), but it's non-obvious and the held merge state dies at disconnect. A first-class begin-merge / resolve / commit-merge flow, or just docs, would help a lot. **Still reproduces** (`dolt.conflicts` empty after the error).
3. ~~**Planner bug on merge tips:** `WHERE commit_hash = HASHOF('')` → `result max1Row iterator returned more than one row`~~ **fixed in 1.3.x — thanks.**
4. **`INSERT … ON CONFLICT DO NOTHING RETURNING`** runs but returns empty rows (`rowCount` is right). Workable, silently wrong for anything that trusts RETURNING. **Still reproduces.**
5. **`DOLT_BRANCH('--remotes')`** silently creates a local branch named `s`; `-r` / `-a` error. `dolt.remote_branches` works instead. **Still reproduces.**
6. **`ALTER TABLE … ADD COLUMN seq bigserial`** — on 1.2.0 every `nextval` failed with `ASSIGNMENT_CAST`; on 1.3.1 the ALTER succeeds but lands **no default at all** (inserts yield NULL) and no reachable sequence. Quieter, not fixed. Workaround: `bigint` + `CREATE SEQUENCE` + `DEFAULT nextval()`. **Changed shape, still broken.**
7. **`ADD COLUMN IF NOT EXISTS`** → `not supported yet`; PL/pgSQL `DO` blocks fail at parse. (`information_schema.columns.column_default` under-reporting got fixed — thanks.) **Still reproduces.**
8. **Remote extremity lifecycle** (design question, not a bug): consumed extremity branches linger on the remote and re-materialize on pull — no push-side branch deletion / advertisement pruning in the file-remote flow. Matrix extremity semantics need one or the other. **Unchanged.**

## Bonus gap :)

History is immutable by design — I've embraced it (redaction works forward, the ref hash covers the redacted form) and I'm documenting "no selective erasure" as a property rather than a bug. But if this ever serves more than homelabbers there's a GDPR angle (not a lawyer), so curious how you think about it — `dolt_squash_history` looks like a hammer.

Curious what you think about it.
Thank you.

Contributor guide

Open the contributing guide

Research direction

The issue does not identify a doltgresql source file or test to change. Start with the linked spikes/RESULTS.md and demo/inspection-tour.md, then reproduce the listed behaviors using the provided demo commands. Done would require a scoped maintainer decision and an agreed implementation or documentation task.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.