GoogleCloudPlatform / GoogleCloudPlatform/knowledge-catalog

Deletion semantics: §5.3 makes removed knowledge indistinguishable from not-yet-written knowledge

Open
#207 20 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
9.2k
Forks
782
Avg merge
6h 36m
Merged PRs (30d)
85

Description

OKF v0.1 has no representation of deletion. Three provisions combine into a loophole worth closing before the format is widely adopted:

1. **§5.3:** "Consumers MUST tolerate broken links — a link whose target does not exist in the bundle is not malformed; it may simply represent not-yet-written knowledge." The tolerance is correct; the *interpretation* is the gap. A dangling link has exactly two histories — the target was never written, or the target was removed — and the spec instructs consumers to assume the innocent one. Removal is thereby spec-invisible.

2. **§7:** `log.md` is optional, and the `**Deprecation**` convention is "a convention, not a requirement." So the only place a removal *could* be recorded is a file that need not exist, in a form that need not be used.

3. **§2/§5.1:** concept identity is the file path. A move or rename — which §5.1 anticipates — destroys the identity entirely. There is no persistent identifier to hang a removal record *on*.

Net effect: a host, producer, or upstream platform can delete concepts from a bundle and every conforming consumer will read the result as a smaller-but-healthy bundle. Silent deletion is fully conformant.

**Why this matters empirically, not hypothetically:** on 2026-06-19 our repository host deleted an 871-DOI scholarly corpus (part of a 1,322,017-record purge). We measured what happened downstream: citation-field retention in affected metadata batches went to 0.00% (n = 1,059 batches), against 100% in controls; graph coverage decayed on a vintage gradient. Platforms delete at scale, and the deletions propagate as *absence of evidence of absence*. A knowledge-exchange format that cannot represent its own losses will launder them into every consuming agent.

**Proposal (minimal, in OKF idiom — all optional, backward-compatible, v0.2-sized):**

1. A `status` frontmatter field with controlled values: `active | deprecated | removed | stub`. Default `active`. `stub` covers concepts whose body is a placeholder or shadow of fuller source material (pairs with the summarization-governance discussion in #53).
2. A reserved **`tombstones.md`** file (same standing as `index.md`/`log.md`): when a concept is removed from a bundle, the producer SHOULD leave a one-line entry — concept ID, date, reason, optional successor link. Removal becomes a marked state instead of a silent diff.
3. **Disambiguated broken-link semantics:** a consumer resolving a dangling link SHOULD check `tombstones.md`; present → the target was removed (surface it as such); absent → not-yet-written stands as the default reading. One lookup, and the two histories separate.
4. An optional stable **`id`** frontmatter field, so identity survives moves/renames and tombstones have something durable to reference.

The design principle, from the archival tradition this borrows from: an archive's (or bundle's) authority doesn't come from being pristine — it comes from being auditable, and auditable means the losses are as legible as the holdings. We've implemented the full version of this apparatus on a 1,086-record corpus (typed completeness states, deletion tombstones, marked-absence artifacts) and are glad to share schemas, decision tables, or a worked PR: https://www.alexanarch.org/s/records/1087/

Related: #53 (producer-side summary policy + consumer receipts — the completeness/`stub` value above is the producer-side declaration that discussion's receipts would test against), #57 (`okf_version` in frontmatter — same mechanism, same place, for a different invariant).

Contributor guide

Open the contributing guide

Research direction

Start by reading §§2, 5.1, 5.3, and 7, then compare the related discussions in #53 and #57. Done means the specification defines backward-compatible status and identity fields, tombstones.md semantics, and how consumers distinguish removed targets from not-yet-written knowledge.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.