GoogleCloudPlatform / GoogleCloudPlatform/knowledge-catalog
Proposal: optional `retrieved_at` / `verified_at` and a source-liveness convention — the container outlives the source
- Dominant language
- TypeScript
- Stars
- 9.2k
- Forks
- 782
- Avg merge
- 6h 36m
- Merged PRs (30d)
- 85
Description
## Summary
§4.1 gives a concept one clock: `timestamp`, "last meaningful change." For bundles whose concepts point at external `resource` URLs, that conflates two things a consumer needs to distinguish — when the *content* changed vs. when a producer last *fetched or verified* it — and the spec has no home at all for a third signal: whether the `resource` still resolves.
This is adjacent to #97 (elevate `timestamp`) and the maintenance-signals discussion in #158, but scoped to the producer-side fields those threads don't cover: retrieval/verification dates and source liveness.
## Evidence (civic public records, where sources rot fast)
We maintain an OKF v0.1 bundle of local-government records for a large county in the US Northeast. Within the bundle's first year we hit, in production:
- **Sources vanishing under live atoms:** a town's public-hearing notices disappeared from its news feed before the year ended. The atom is fine; its `resource` is gone. Nothing in the format records or reveals that.
- **Verification as a distinct event:** libraries post board minutes months late; a fact can be *re-verified unchanged* long after its content `timestamp`. A consumer deciding whether to trust a figure needs "verified last week" vs. "verified last year," which `timestamp` cannot express without destroying its content-change meaning.
- **Liveness is checkable but nontrivial:** we run a sweep that GETs every `resource` in the bundle (~160 URLs). Getting truthful answers required a browser user-agent, per-host politeness delays, and 429 backoff — civic-portal hosts throttle and UA-block. It's exactly the kind of shared problem a documented convention (plus a reference tool) would keep every producer from re-solving.
## Proposal (all optional, all additive under §4.1 / §11)
1. `retrieved_at` — when the producer last fetched the `resource`.
2. `verified_at` — when a human or pipeline last confirmed the concept still matches its source.
3. A `source_status` convention — last-checked date + last observed HTTP state for the `resource` (e.g. `{checked: 2026-07-02, state: ok | dead | blocked}`).
Consumers that don't know the fields ignore them; nothing about §9's permissive-consumption contract changes. A v0.1 bundle stays conformant.
We have a working liveness checker (Python, no dependencies beyond the standard library + requests) built for this bundle and would be glad to contribute it as a reference implementation if there's interest.
Contributor guide
Research direction
Start with §4.1, §9, and §11, then compare adjacent issues #97 and #158. Evaluate the three optional producer-side signals and the proposed liveness convention, including the offered Python checker; done means the project reaches a documented decision on scope and compatibility.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100