GoogleCloudPlatform / GoogleCloudPlatform/open-knowledge-format

Deletion semantics: a removed concept and a concept that was never written are indistinguishable to a consumer enumerating a bundle

Open
#11 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
467
Forks
35
Avg merge
9m
Merged PRs (30d)
1

Description

OKF handles **deprecation** well. `bundles/acme_retail/metrics/gross-margin-legacy.md` carries `status: deprecated`, stays in place, remains listed in `metrics/index.md`, and its retirement is narrated in `log.md`. A consumer enumerating the bundle sees the concept, sees that it is retired, and can follow the body link to its successor. That is the right shape — and the spec's own gloss says why: `deprecated` is "kept for links and history; no longer current."

There is no standardized equivalent for **removal**. If that file had been deleted rather than retired, a current enumeration of the bundle would contain no concept document for it and, in an index kept in sync with current contents, no listing. The only surviving indication might be a line in `log.md` saying it was removed. A consumer enumerating current concepts therefore sees nothing — and cannot distinguish that state from a concept that was never written at all.

The spec is silent here. `SPEC.md` defines no `tombstone`, `presence`, `removed`, or `never_landed` convention; the lifecycle enum is `draft | stable | deprecated`, which covers the case where the body is kept and not the case where it is gone. Nor does `log.md` supply a machine-readable substitute: it is a Log-typed markdown document whose entries are prose, and the spec prescribes no structured absence assertion within it.

## This is demonstrated, not hypothesised

Two independently built implementations — a producer emitter and a consumer checker, different authors, neither aware of the other's internals — reproduce the collision on their own published artifacts:

A created and present → present
B created, then removed; removal fully recorded → never_landed
C creation asserted, never landed → never_landed

B and C are observationally identical. The producer did **not** lose the removal: it recorded stable id, presence, date, reason, recording authority and successor. The consumer correctly reported an asserted target that does not resolve. **The loss is at their interface**, because the removal was preserved in prose rather than on the surface the consumer enumerates.

Necessity and sufficiency were tested rather than asserted. Add a surviving absence record to the machine-readable discovery surface and make one lookup before assigning `never_landed`: B resolves as `removed`, C stays `never_landed`, and no other case changes. Strip that record while leaving the prose record fully intact, and B and C collapse again.

Full reproducibility package — pinned commits, clean-clone reproduction, both runs, hashes: https://www.alexanarch.org/datasets/deletion-semantics-cross-implementation-test/

## Proposal

Minimal, and an extension of an idiom OKF already has rather than a new mechanism:

**An absent concept retains an entry on the discovery surface, carrying its stable identity and its absence state.** In OKF's terms, `index.md` retains a line for the removed concept — with no link, or a link to a stub — and the state is `removed` or `never_landed` alongside the existing `deprecated`.

The payload tested was:

```yaml
id:
presence: removed | never_landed
date:
reason:
recorded_by:
successor: # optional
successor_kind: # optional
```

`recorded_by` matters: it makes an absence attributable rather than merely observed.

The normative requirement, stated independently of carrier:

> A surviving absence assertion must be exposed on a machine-readable discovery surface that ordinary consumers consult. Preserving the same information only in prose is insufficient.

`index.md` is OKF's natural carrier because it is already what consumers enumerate, and `log.md` is already where the prose goes — prose alone is what fails. Another producer could satisfy the requirement with a structured log event or a reserved absence file. Note that `index.md` is optional in the spec, so the requirement should attach to whatever discovery surface a bundle publishes rather than to `index.md` by name.

## What I am not proposing

Not a new file type. Not a change to `log.md`, which is doing its job. Not the specific field names above, which are simply what was tested. Not retention of removed *content* — only of the assertion that something was removed, keyed to stable identity.

## Conformance material

A 117-case conformance corpus encodes the requirement, including the A/B/C trio with B split by whether the absence is machine-visible: https://www.alexanarch.org/datasets/deletion-conformance-fixture/

It is discriminating rather than descriptive: a reference consumer that omits the lookup produces two false positives and fails the must-not-mark traps; the same consumer with the lookup produces none. CC0.

Prior design discussion, and the origin of this work, is at knowledge-catalog#207. That thread is the laboratory record; this issue is where the normative question belongs now that the OKF surface has moved here.

Contributor guide

Open the contributing guide

Research direction

Start with SPEC.md, then inspect the discovery surface represented by metrics/index.md and the history recorded in log.md. Compare the current lifecycle and enumeration rules with the linked A/B/C conformance corpus; done means the specification distinguishes machine-visible removal from never-landed concepts without requiring removed content to remain.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.