GoogleCloudPlatform / GoogleCloudPlatform/knowledge-catalog

Budget-aware assembly for OKF: optional edge priority levels

Open
#139 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
9.2k
Forks
782
Avg merge
6h 36m
Merged PRs (30d)
85

Description

Thanks for publishing OKF. The "format, not platform" framing resonates strongly with us.

We maintain [CodeMemory](https://github.com/ZaVang/CodeMemory), a single-owner, cross-project memory system for agents. It comes from a different domain than data catalogs: distilled judgments, decisions, preferences, and project context, written mostly by agents and consumed by many agents across environments. We started it before seeing OKF, so the convergence was striking: one concept per `.md` file, YAML frontmatter as the queryable interface, markdown body as the implementation, git for history, and lexical search mainly to find an entry point.

I read #86, #101, and #120 before opening this. CodeMemory overlaps with several topics already discussed there: typed-ish relationships, frontmatter relationship indexes, rationale trails, and health checks. I wanted to raise one layer I do not see covered yet.

## The missing layer: assembly under a context budget

For machine-consumed bundles, the hard part is not only representing links. It is deciding, at consume time, what to actually place in the context window when the whole bundle will not fit.

CodeMemory's premise is that agent memory can be organized like code: each concept declares its dependencies explicitly, the way a source file declares its imports. That is what turns context construction into a single `build` — resolve the declared dependencies into a closure and assemble exactly what an entry point needs, deterministically rather than by similarity search:

> entry concept -> dependency closure -> topological order -> trim to a token budget -> render

The assembler keeps the entry and its must-read dependencies, while lower-priority material can degrade to each concept's `description` or summary.

This is intentionally consumer-side, and I understand OKF does not want to prescribe storage, serving, or query infrastructure. I do not think the spec should define an assembler. But a budgeted assembler benefits from one small piece of optional information that OKF does not currently encode.

## Proposal: relationship `level`, orthogonal to relationship `kind`

#86 and #101 discuss the *kind* of a relationship: `depends-on`, `joins-with`, `references`, and so on. A budgeted assembler needs a different axis: the *level* of the relationship, meaning what a consumer should do with the target when context is scarce.

Possible levels:

- `required`: include fully when assembling A; A is likely to be misunderstood without it.
- `recommended`: include if budget allows; degrade to `description`/summary under pressure.
- `related`: keep as a link; do not inline by default.

The important distinction is:

> `level` is not `kind`.

A `depends-on` edge can still be low-priority for a particular assembly task, while an otherwise ordinary prose mention may be required to understand the current concept. `kind` describes the semantic relationship; `level` describes the assembly priority.

This could remain fully optional and backward-compatible. It might be represented through the markdown link title convention discussed in #101, or through a frontmatter relationship list like the one discussed in #120. Consumers that do not care about assembly can ignore it.

## Two small related notes

First, if consumers may degrade a concept to only its `description`, it may be worth adding a short authoring recommendation: write `description` so the concept remains useful when that is all a budgeted consumer keeps. We treat this as the concept's signature.

Second, in CodeMemory we found it important to keep "must read to understand this" separate from "source/provenance for this claim." The former participates in assembly; the latter belongs in citations. Mixing them tends to pull source material into the context closure when the consumer only needed provenance.

Happy to share the assembly algorithm or contribute a small reference consumer if that would be useful. Also glad to split this into smaller issues if maintainers prefer.

Thanks again for the spec.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing issues #86, #101, and #120, especially their discussions of relationship kinds, link titles, and frontmatter relationship lists. Decide whether an optional assembly level belongs in the specification, how it should remain distinct from kind and citations, and what backward-compatible representation would make the proposal complete.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.