waldronlab / waldronlab/agent-protocol-standard

Specify the registry schema and define trust_tier semantics

Open
#22 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3
Forks
1
Avg merge
14h 8m
Merged PRs (30d)
23

Description

PROTOCOL_STANDARD.md specifies protocol.md and nothing else. The registry format is written down nowhere.

registry.yaml uses registries:, index_url, trust_tier, approved_by, and registered_date, but no document defines any of them. They exist only as one example file and as prose in skills/protocol-runner/SKILL.md. No tooling reads the file either — validate-protocol.R and generate-protocols-yaml.R never touch it. The file the entire federation's trust model rests on is both unspecified and unvalidated.

trust_tier is the sharpest instance

The runner ranks candidates by trust_tier descending (SKILL.md:36) and prints it in the Method Provenance block (SKILL.md:84), but nothing states:

  • the range of the scale,
  • whether higher means more trusted,
  • what qualifies a repository for a given tier, or
  • who assigns it and on what evidence.

The only value in existence is trust_tier: 4, and nobody can say what 4 means. SKILL.md:143 then says trust scores are "reserved for a future release" — so the runner both ranks by the field and describes it as unimplemented. One of those is wrong.

This is not cosmetic. trust_tier is how the runner chooses between two protocols of the same name from different labs. An undefined scale means that choice is arbitrary while appearing principled, and it is displayed to the user as if it were evidence.

Proposed
  1. A Registry Schema section in PROTOCOL_STANDARD.md: every field, required vs. optional, types, and meaning. ADR 0006 already establishes the load-bearing rule — the unit of trust is the repository, not the protocol or the domain — so that belongs here too.
  2. Define trust_tier explicitly: the scale, its direction, and the criteria for each tier. If the criteria are not yet settled, define the scale and mark tier assignment as provisional rather than leaving the field undefined but active.
  3. Reconcile SKILL.md:143 with the ranking rule at :36.
  4. Validate registry.yaml in CI. A malformed registry breaks discovery for every consumer at once, and it is currently the only unvalidated file in the system.
Why first

waldronlab/agent-protocol-standard#23, #24, and #25 all extend the registry schema. None of them can be specified against a schema that does not exist, and the trust questions they raise — what tier does an unregistered node have, what does an imported registry inherit — are unanswerable until trust_tier means something.

Deferred: hierarchical registries (registries of registries)

Recorded as ADR 0008, so the reasoning has a home that survives this issue closing — its own revisit trigger is this issue landing, which would otherwise bury the record at the moment it becomes relevant again.

Summary: registry.yaml stays a flat list. Federation happens by choice of root (SKILL.md:28 already lets an agent be pointed at any repository's registry), and the private/pre-publication case is left to #23's proposed ad-hoc attachment. Revisit condition unchanged: enough nodes that a flat registry is an actual curation burden, and trust_tier has semantics.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with PROTOCOL_STANDARD.md, registry.yaml, skills/protocol-runner/SKILL.md, and ADR 0006; compare the documented registry fields with the runner's ranking and provenance references. Read validate-protocol.R and generate-protocols-yaml.R, then inspect the existing CI configuration for validation entry points. Done means the registry schema and provisional or final trust_tier semantics are documented, SKILL.md is consistent, and malformed registry.yaml is checked in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
r, yaml
Domain
ci-cd, documentation, tooling
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.