mozilla / mozilla/markfluence

actionlog: compaction, and the attachment-* history lines nothing reads yet

Open
#162 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
2
Forks
0
Avg merge
2h 8m
Merged PRs (30d)
49

Description

Two pieces of internal/actionlog that #149 named and _plans/041_page-moved-detection.md deferred. Both are about the log as history rather than the log as a base, which is why they went together and why neither blocked #149: the base check reads only the last ok line for a file, and it works today.

Compaction

#149 promises a compaction setting rather than a truncation, because the history has uses beyond the base — #148's status view, and answering "why did this page change?" — so bounding growth must not mean throwing the history away.

The growth is real but slow: a line is ~180 bytes, so a 200-page tree published daily is ~13 MB of uncommitted local state a year. Nothing breaks at that size; Log.read is linear and runs once per root per run via actionlog.Cache. So this is eventually-real, not 1.0.0-real.

What it needs deciding:

  • What the setting is and where it lives. markfluence.yaml's settings are unknown-key-fatal by design (internal/project/config.go), which cuts both ways: adding a key later is cheap, but a project file written for a newer markfluence is refused by an older binary. That is the established trade and needs no new answer, only a name.
  • What compaction keeps. Keeping the last N lines per file is the obvious rule and is not the same as keeping the last N lines, since a batch writes one line per page. Keeping everything newer than a duration is the other shape.
  • Rewriting a file whose appends are unsynchronized. Every writer today is a single O_APPEND write with no lock, deliberately (the project.SetPageEntry/client.SetContentProperty posture — a lock file brings stale-lock handling to a verb a person invokes by hand). Compaction is a read-modify-write over the whole file, which is a different animal: a concurrent append during the rewrite is lost. Renaming within .markfluence/ is safe from git's point of view (the planted .gitignore holds *), so the temp-file-and-rename shape is available.
  • When it runs. Opportunistically on a write past a size threshold is the cheap answer; a markfluence verb for it is the explicit one.

One consequence already recorded in the plan: compaction adds another way to lose an entry, and that case is the same row as "no line naming this file" — unknown base, publish, silent. It needs no new semantics, which is what makes it safe to add after the fact.

attachment-* history lines

attachment-upload and attachment-download are #149's "history only" rows: they pair a local file with a page, so they could log, but nothing would read what they wrote.

They were deferred on a specific measurement rather than on effort: an attachment upload does not bump the page version, so an attachment write cannot invalidate a base and cannot participate in either check. A logged attachment-upload line would be a row no reader consults.

So this is machinery for a consumer to ask for, and the consumer is #148 — a status view that says "this page's diagram was replaced on Tuesday" wants exactly these rows. Filing it now so the reasoning survives; it should land with, or after, whatever reads it.

Deferred alongside these and not part of this issue: info reporting the base ("last published by markfluence at v44"), which is #148's business.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with internal/actionlog and internal/project/config.go, then read _plans/041_page-moved-detection.md and the related #148 and #149 issues. Resolve the compaction setting, retention rule, rewrite safety, and trigger, and decide when attachment-* rows should be written. Done means an agreed design and implementation that preserves the existing base semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.