Bundle-local registry for external OKF references

# Proposal: Bundle-local registry for external OKF references

## Framing

This proposal is written based on my current reading of the OKF v0.1
draft SPEC and is offered as a **contribution to the discussion**,
not as a finished specification. It is entirely possible that I am
identifying a problem where none exists; the OKF use case as the
maintainers see it may differ from mine, or the gap I describe may
already be intended to be handled outside the SPEC.

My reading of OKF: it is meant to be a **self-contained format** for
providing information to AI agents, which typically consume that
information from files on disk. From that vantage point, the use
case that motivates this proposal is that such bundles will often
need to include information from **other OKF bundles** without
giving up self-containment.

If the maintainers' framing of OKF differs; for example, if bundles
are envisioned as strictly leaf artefacts and cross-bundle knowledge
is expected to be handled by the consuming service or agent; then
the mechanism proposed below is not needed and I welcome that
redirection.

The rest of this document assumes that reading is at least plausible
and proposes a minimal, backwards-compatible mechanism.

## Motivation

The OKF v0.1 SPEC establishes bundles as **self-contained**
hierarchical collections of markdown documents (§3). Absolute links
start with `/` and resolve relative to the bundle root (§5.1). This
is intentional and gives OKF its clean, portable substrate.

In practice, however, a bundle almost always **references knowledge
that lives in another bundle**. A private organisational catalog may
want to cite entries from `crypto_bitcoin` or `ga4` under
`GoogleCloudPlatform/knowledge-catalog`; a research bundle may want
to pull in concepts from a domain reference bundle it doesn't own.

The SPEC currently offers no mechanism for such references beyond
filesystem-relative paths, which break the self-contained property
and depend on how the consuming project happens to lay bundles out on
disk.

This proposal adds a **minimal, backwards-compatible mechanism** for
external references while preserving OKF's self-containment.

## Design

### 1. Bundle-local registry file

A bundle MAY contain a file `okf-registry.yaml` at its root. Its
presence declares that the bundle imports one or more external OKF
bundles at specific **mount paths** within its own tree.

**Filename convention:** `okf-registry.yaml` (fixed; discoverable by
name).

**Schema:**

```yaml
version: 1
mounts:
:
source:
ref:
subpath:
checksum:
```

**Example:**

```yaml
version: 1
mounts:
external/ga4:
source: git+https://github.com/GoogleCloudPlatform/knowledge-catalog.git
subpath: okf/bundles/ga4
ref: main
external/bitcoin:
source: git+https://github.com/GoogleCloudPlatform/knowledge-catalog.git
subpath: okf/bundles/crypto_bitcoin
ref: main
vendor/legal-base:
catalog: https://acme.internal/okf-catalog.yaml
catalog_query: okf-catalog-v1
bundle: base
ref: v2.1.0
```

Mount paths are ordinary subpaths within the bundle. Bundle authors
MAY group them under a common prefix (e.g. `external/`) or place
them anywhere.

**Multi-source (mirrors and fallbacks).** Any URL-valued field
(`source`, `catalog`) MAY be given as either a single string or a
YAML sequence of strings. Tools SHOULD try entries in order and use
the first that resolves successfully:

```yaml
mounts:
external/ga4:
source:
- git+https://github.com/GoogleCloudPlatform/knowledge-catalog.git
- git+https://mirror.acme.internal/knowledge-catalog.git
subpath: okf/bundles/ga4
ref: main
```

### 2. Provider types and catalog resolution

A mount's `source` (or `catalog:` reference) is resolved to actual
bundle content by the OKF manager tool. Two resolution paths are
proposed.

#### 2.1 Direct source

`source` names a concrete location for the bundle:

| Prefix | Meaning |
|-------------------------------------|-----------------------------------------------------------------------------|
| `git+https://.git` | Git repository over HTTPS. `ref` = branch, tag, or commit; `subpath` = bundle path inside the repo. |
| `git+ssh://.git` | Same, over SSH. |
| `https://.zip` (or `.tar.gz`) | Downloadable archive; `checksum` recommended for integrity. |

Example:

```yaml
mounts:
external/ga4:
source: git+https://github.com/GoogleCloudPlatform/knowledge-catalog.git
subpath: okf/bundles/ga4
ref: main
```

#### 2.2 Catalog reference

Instead of a direct source, a mount MAY delegate resolution to an
**external catalog**. Real-world knowledge catalogs come in many
shapes: some are static YAML files, others are live services (e.g.
Google Knowledge Catalog / Dataplex, or private data-catalog
products with proprietary APIs). This proposal therefore
standardises the **resolution mechanism** but deliberately does
**not** fix a single catalog file format or protocol.

A catalog reference declares three things:

- **`catalog:`** — the URL of the catalog (or an array of URLs).
- **`catalog_query:`** — the name of the query adapter that knows
how to interpret the catalog and look up a bundle by name.
- **`bundle:`** — the name or ID of the bundle to look up inside
that catalog.

Example:

```yaml
mounts:
external/ga4:
catalog: https://acme.internal/okf-catalog.yaml
catalog_query: okf-catalog-v1
bundle: ga4
ref: main # optional, overrides the catalog's default
```

The tool selects the query adapter identified by `catalog_query`,
uses it to fetch the catalog at `catalog:` and look up `bundle:`,
then resolves the result to a concrete source and materialises the
bundle at the mount path.

**Suggested default adapter name:** `okf-catalog-v1`. The concrete
schema (or service protocol) behind this name is intentionally left
open so that alignment can happen upstream,
ideally in coordination with the Knowledge Catalog project and any
other prospective catalog backends.

Consumers unaware of a specific query adapter SHOULD fail cleanly
("unsupported catalog query") rather than attempt best-effort
resolution.

#### 2.3 Self-containment

Both resolution paths keep the bundle self-describing: everything
needed to resolve an external mount is inside `okf-registry.yaml`.
A direct source names the concrete location; a catalog reference
names the catalog URL, the query adapter to interpret it, and the
bundle key to look up. Nothing depends on tool-side configuration
or on well-known names outside the bundle.

Consumers unaware of a specific provider form or query adapter
SHOULD ignore the mount gracefully.

### 3. Linking to external content

Within the importing bundle, references to external content are
**ordinary bundle-absolute markdown links** (§5.1):

```markdown
See [GA4 Events](/external/ga4/tables/events.md) for schema.
```

No new URI scheme, no new link form. The link is valid OKF because
the mount path is part of the bundle's tree from the consumer's
perspective.

### 4. Tool responsibilities

An OKF manager tool implementing this proposal MUST:

1. Read `okf-registry.yaml` at bundle load time.
2. Make the declared external bundles available at their mount paths
before consumers traverse the bundle.
3. Ensure the mount targets are read-only (writes belong in the
source, not the mount).

**Phase 1 implementation:** materialise each mount as a read-only
filesystem copy fetched from `source` at the specified `ref`. Cache
under the mount path.

**Phase 2 (optional):** mount via FUSE (or an equivalent user-space
filesystem) so external content is streamed on demand rather than
copied. The mount path stays the same; only the implementation
underneath changes.

### 5. Version-control hygiene

Mount targets SHOULD be excluded from the bundle's version control
(e.g. listed in `.gitignore`). The `okf-registry.yaml` (including
`source`, `ref`, and optional `checksum`) is the reproducible
declaration; the materialised bytes are a cache.

### 6. Self-containment is preserved

From an OKF consumer's perspective, the bundle **is** self-contained:
every referenced path resolves within the bundle tree.

The registry is a **producer-side** agreement about how the tree
gets populated. This is fully compatible with the existing v0.1
SPEC — consumers that ignore `okf-registry.yaml` still see a valid,
self-contained bundle tree once the manager tool has populated the
mounts.

## Open questions

- **Nested bundles:** if a mounted bundle contains its own
`okf-registry.yaml`, does the tool recurse? Proposed: yes, with
cycle detection.
- **Read-only enforcement:** MUST or SHOULD? Proposed: SHOULD;
producers may explicitly override for local development.
- **Registry file format:** YAML only, or should JSON be permitted?
Proposed: YAML only for consistency with frontmatter conventions.
- **Catalog format(s) behind `catalog_query`:** the concrete schema
or service protocol identified by adapter names such as
`okf-catalog-v1` is deliberately left open by this proposal. Real
catalogs today span static YAML files and live services (e.g.
Google Knowledge Catalog / Dataplex). The community should align
on one or more canonical formats, each with its own adapter name.
This proposal only fixes the *mechanism* by which a registry
entry names an adapter and passes it a bundle key.
- **Multi-source semantics:** for array-valued URL fields, tools
SHOULD try entries in order — but should they fail fast on the
first success, race in parallel, or verify all yield equivalent
content? Proposed: sequential try-in-order, first success wins;
checksums (where present) SHOULD be verified to catch mirror
divergence.

## Backwards compatibility

Fully backwards-compatible. `okf-registry.yaml` is optional; bundles
without it behave exactly as before. Consumers unaware of the
proposal see mounts as ordinary subdirectories (once populated) and
process them as regular OKF content.

## Reference implementation

Not yet available. This proposal is the minimum specification
needed before implementation begins.

No implementation files or tests are identified, and the proposal states that a reference implementation is not yet available. Start by reviewing the OKF v0.1 SPEC sections cited in the proposal and resolving the open design questions; done means the registry behavior is agreed and an implementation scope is established.

Written by the indexing model from the issue text.