GoogleCloudPlatform / GoogleCloudPlatform/knowledge-catalog
Deletion semantics: §5.3 makes removed knowledge indistinguishable from not-yet-written knowledge
- Dominant language
- TypeScript
- Stars
- 9.2k
- Forks
- 782
- Avg merge
- 6h 36m
- Merged PRs (30d)
- 85
Description
OKF v0.1 has no representation of deletion. Three provisions combine into a loophole worth closing before the format is widely adopted:
1. **§5.3:** "Consumers MUST tolerate broken links — a link whose target does not exist in the bundle is not malformed; it may simply represent not-yet-written knowledge." The tolerance is correct; the *interpretation* is the gap. A dangling link has exactly two histories — the target was never written, or the target was removed — and the spec instructs consumers to assume the innocent one. Removal is thereby spec-invisible.
2. **§7:** `log.md` is optional, and the `**Deprecation**` convention is "a convention, not a requirement." So the only place a removal *could* be recorded is a file that need not exist, in a form that need not be used.
3. **§2/§5.1:** concept identity is the file path. A move or rename — which §5.1 anticipates — destroys the identity entirely. There is no persistent identifier to hang a removal record *on*.
Net effect: a host, producer, or upstream platform can delete concepts from a bundle and every conforming consumer will read the result as a smaller-but-healthy bundle. Silent deletion is fully conformant.
**Why this matters empirically, not hypothetically:** on 2026-06-19 our repository host deleted an 871-DOI scholarly corpus (part of a 1,322,017-record purge). We measured what happened downstream: citation-field retention in affected metadata batches went to 0.00% (n = 1,059 batches), against 100% in controls; graph coverage decayed on a vintage gradient. Platforms delete at scale, and the deletions propagate as *absence of evidence of absence*. A knowledge-exchange format that cannot represent its own losses will launder them into every consuming agent.
**Proposal (minimal, in OKF idiom — all optional, backward-compatible, v0.2-sized):**
1. A `status` frontmatter field with controlled values: `active | deprecated | removed | stub`. Default `active`. `stub` covers concepts whose body is a placeholder or shadow of fuller source material (pairs with the summarization-governance discussion in #53).
2. A reserved **`tombstones.md`** file (same standing as `index.md`/`log.md`): when a concept is removed from a bundle, the producer SHOULD leave a one-line entry — concept ID, date, reason, optional successor link. Removal becomes a marked state instead of a silent diff.
3. **Disambiguated broken-link semantics:** a consumer resolving a dangling link SHOULD check `tombstones.md`; present → the target was removed (surface it as such); absent → not-yet-written stands as the default reading. One lookup, and the two histories separate.
4. An optional stable **`id`** frontmatter field, so identity survives moves/renames and tombstones have something durable to reference.
The design principle, from the archival tradition this borrows from: an archive's (or bundle's) authority doesn't come from being pristine — it comes from being auditable, and auditable means the losses are as legible as the holdings. We've implemented the full version of this apparatus on a 1,086-record corpus (typed completeness states, deletion tombstones, marked-absence artifacts) and are glad to share schemas, decision tables, or a worked PR: https://www.alexanarch.org/s/records/1087/
Related: #53 (producer-side summary policy + consumer receipts — the completeness/`stub` value above is the producer-side declaration that discussion's receipts would test against), #57 (`okf_version` in frontmatter — same mechanism, same place, for a different invariant).
Contributor guide
Research direction
Start by reading §§2, 5.1, 5.3, and 7, then compare the related discussions in #53 and #57. Done means the specification defines backward-compatible status and identity fields, tombstones.md semantics, and how consumers distinguish removed targets from not-yet-written knowledge.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100