GoogleCloudPlatform / GoogleCloudPlatform/knowledge-catalog

OKF Corpus is not bitemporal

Open
#253 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
9.2k
Forks
782
Avg merge
6h 36m
Merged PRs (30d)
85

Description

OKF is distributed as a snapshot, a directory, the consuming engine sees files.
OKF has no way to express what a corpus asserted at a given moment, or what it has stopped asserting, there is no valid_from or valid_until only stale_after
A corpus continuously written by agents that prune their own errors converges toward looking reliable regardless of whether it is. Deleted files are invisible to the engine, so every evaluation run against the corpus is contaminated by the removals. you are measuring the survivors.
There is currently no way to record that a claim was believed, tested, and found wrong. status: deprecated points from an old definition toward a new one, but carries no reason. The knowledge that a particular formulation fails, and why, is destroyed at exactly the moment it becomes useful. An agent regenerating the concept next quarter has nothing to stop it re-deriving the same error.

An error that gets pruned is knowledge destroyed, not corpus hygiene.

log.md is reserved and nominally records history, but it is a byproduct: prose entries, written incidentally, with no guarantee that anything is written when a concept is removed. A history mechanism built by scraping it inherits that unreliability. If the evolution path of a concept is worth preserving, it needs to be a deliberate artifact with its own conformance rules.

The only way this problem is addressed in finance is by introducing unique ids, check the openfigi initiative https://www.openfigi.com/about/features

In short bitemporal data requires :
- a unique id source ( doesn't need to be universal but unique to the corpus )
- ways to store and access 'not curent' version of a record -> valid from date A to date B ( current being date B as null or 100 years from now )

Contributor guide

Open the contributing guide

Research direction

Start by reviewing log.md and how the corpus currently represents snapshots, directories, files, stale_after, and deprecated concepts. Define the corpus-specific unique ID and the deliberate history artifact with conformance rules. Done means the engine can access superseded records and represent valid_from and valid_until rather than silently losing removed claims.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.