BOHICA-LABS / BOHICA-LABS/vsdd-factory

Branch-protection verification via /branches/{branch} is unreliable on private non-Enterprise repos

Open
#348 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Summary

The vsdd-factory orchestrator skill and devops-engineer agent (P3-01 step) verify branch protection by reading the \`protected\` field on \`GET /repos/{owner}/{repo}/branches/{branch}\`. On private repos owned by a GitHub organization without an Enterprise plan, that endpoint returns \`protected: true\` with a fully-populated \`protection\` object even when no ruleset and no legacy branch protection actually exists. This is a false-positive verification path that lets the autonomous pipeline advance past P3-01 with main unprotected.

## Process-gap class

PG-7 (in the running internal numbering on akey). Same family as PG-5 (#346) and PG-6 (#347) — orchestrator workflow steps trusting a single GitHub endpoint without cross-checking.

## Concrete reproducer (just observed on akey)

\`\`\`
$ gh api repos/ArcavenAE/akey/rulesets
[]

$ gh api 'repos/ArcavenAE/akey/rules/branches/main?includes_parents=true'
[]

$ gh api repos/ArcavenAE/akey/branches/main/protection
{"message":"Not Found","documentation_url":"...","status":"404"}

$ gh api repos/ArcavenAE/akey/branches/main | jq '{protected, protection}'
{
"protected": true,
"protection": {
"enabled": true,
"required_status_checks": {
"enforcement_level": "non_admins",
"contexts": ["fmt","clippy","build","test","deny"],
"checks": [...]
}
}
}
\`\`\`

Repo is private, owner is an Organization, no Enterprise plan. There are no rulesets at repo level, none inherited from org, and no legacy protection. Yet \`/branches/main\` reports protected:true with all 5 required checks. I closed the admin-action tracking issue on that read, then a human spotted the discrepancy.

## Why \`/branches/{branch}\` is the wrong source of truth

Three independent endpoints are authoritative for branch protection:

| Endpoint | What it covers | Reliable? |
|---|---|---|
| \`GET /repos/.../rulesets\` | repo-level rulesets | Yes, but org-level rulesets are NOT in this list |
| \`GET /repos/.../rules/branches/{branch}?includes_parents=true\` | every rule that applies (repo + org), evaluated at the branch | **Yes — single source of truth** |
| \`GET /repos/.../branches/{branch}/protection\` | legacy branch protection only | Yes for legacy; 404 when ruleset-based |
| \`GET /repos/.../branches/{branch}\` (\`protected\` field) | summary view | **Unreliable on private non-Enterprise repos** |

The summary field is a derived/cached view that does not match ground truth in this configuration.

## Proposed fix

In the orchestrator-skill / devops-engineer / state-manager step that verifies P3-01 completion:

1. Stop trusting \`branches/{branch}.protected\` as a verification signal.
2. Verify by calling \`GET /repos/.../rules/branches/{branch}?includes_parents=true\` and asserting the returned array is non-empty AND contains rules whose \`type\` matches the required set (e.g. \`required_status_checks\`, \`required_linear_history\`, \`non_fast_forward\`, \`deletion\`).
3. If that endpoint returns \`[]\`, also check \`branches/{branch}/protection\` for the legacy case before declaring P3-01 complete. Both empty = NOT PROTECTED.
4. Update the gate-template / human-action issue body (PG-5 context) to instruct the admin to verify via the rules endpoint, not via the summary.

## Operational impact

Without this fix, a fully-autonomous greenfield pipeline running against a private org repo will pass the P3-01 verification gate even if the admin never actually applied the protection (or applied it incorrectly), and then proceed to merge story PRs via pr-manager with no required-check enforcement on main. Defeats the entire purpose of P3-01.

## Reference

- Filing repo (akey): https://github.com/ArcavenAE/akey/issues/6 (just reopened with the correction)
- Sibling defects in this family: #343 (PG-1), #346 (PG-5), #347 (PG-6)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.