sillsdev / sillsdev/harmony

[claude] Add an EntityTouches ledger table

Open
#120 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
14
Forks
4
Avg merge
3d 21h
Merged PRs (30d)
3

Description

[Claude, autonomous]

Record every entity touch in a never-pruned local table, so Snapshots is allowed to stay incomplete. Derived state like snapshots are: local only, never synced, rebuilt by replay.

EntityTouches
  CommitId         TEXT NOT NULL     -- FK Commits.Id, cascade delete
  EntityId         TEXT NOT NULL
  ChangeIndex      INTEGER NOT NULL  -- the change that caused the touch; for a cascade, the causing change
  References       TEXT NOT NULL     -- JSON guid array, as Snapshots.References
  EntityIsDeleted  INTEGER NOT NULL
  PRIMARY KEY (EntityId, CommitId, ChangeIndex)

Why:

  • Rebuild an entity's graph at a past commit from the relationship ids, instead of materialising the whole project at that commit.
  • Exhaustive entity history, including the cascade touches from SnapshotWorker.MarkDeleted that no ChangeEntity names.
  • Cheaper replay for a point-in-time read: resume to the entity's last ledger entry before X rather than to X.

The graph rebuild is the biggest win and the least proven of the three, so a PR closing this should demonstrate it with tests at acceptable performance and land the core machinery that feature needs, not just the table.

Per change rather than per commit because FW Lite's entity history is already a flat per-change list keyed on (commitId, changeIndex). Cascade rows there have no change index today, default to 0, collide with the commit's first change and load the wrong change's context; giving them the causing change's index fixes that and lets the view say why an entity was touched.

No Kind column: created is the entity's first row, deleted is EntityIsDeleted flipping true, revived is it flipping back, and cascade-vs-direct is an anti-join on ChangeEntities.

Costs about 8% of a project database, roughly 110 bytes per row. No index beyond the key: adding one on CommitId would cost another 79 bytes per row and nearly double the table, and the only query that filters by commit is the delete that trims the tail during a rewind, measured at 53 ms without it against 21 ms with it, next to a replay that costs far more. Add it later if something needs it. Storing ids as 16 raw bytes instead of text would roughly halve the table, which isn't worth having one table spell ids differently from the rest of the schema. Measurements and the rest of the reasoning: https://claude.ai/artifact/MQzjioQNp9qUYdf3hrP5Cq?sk=QR_R5YFXJaxroVEhjslL6w

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing Snapshots and FW Lite entity-history handling, especially SnapshotWorker.MarkDeleted and the ChangeEntity paths mentioned in the issue. The work is done when the EntityTouches ledger and its cascade change indexes support graph reconstruction at a past commit, with tests demonstrating acceptable replay performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.