feat(readiness): Org-level and GitHub settings assessment pillar
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1.1k
- Forks
- 94
- Avg merge
- 8d 3h
- Merged PRs (30d)
- 7
Description
Problem
AgentRC's readiness assessment currently operates at the repository level — it checks for instruction files, lockfiles, CI configs, and other repo-scoped signals. But during customer engagements, field teams also need to assess organization-level and GitHub-settings-level readiness that determines whether AI tooling can be effective across an org's repos.
These signals are queryable via the GitHub API but aren't captured by any AgentRC pillar today. Without them, a repo can score well on readiness while the org-level configuration actually prevents effective Copilot usage.
Signals to assess
GitHub org settings (via API)
- Copilot enablement: Is GitHub Copilot enabled for the org? Which policy (all members, selected teams, disabled)?
- Copilot Chat in IDE: Enabled or restricted?
- Code referencing: Public code filter on or off?
- Content exclusions: Are there Copilot content exclusion patterns that might affect the assessed repos?
Repository-level GitHub settings
- Branch protection rules: Are default branch protections configured? (Required reviews, status checks, signed commits)
- Code scanning / CodeQL: Enabled? Which languages covered?
- Dependabot: Enabled for security alerts? Version updates configured?
- Secret scanning: Enabled with push protection?
- Actions permissions: Which actions are allowed? Are reusable workflows enforced?
Cross-repo patterns
- Template repos: Does the org use template repos that include AI instruction files?
.githubrepo: Is there a community health.githubrepo with default instruction files?- Inner source config: Are there org-level
.github/copilot-instructions.mdor instruction packages?
Proposal
New readiness pillar: org-settings
Add an optional pillar that queries org/repo settings when a GitHub token is available:
agentrc readiness --scope=org --org=contoso
Or assess org-level settings for the current repo's org:
agentrc readiness --include-org
Criteria examples
| Criterion ID | What it checks | Impact |
|---|---|---|
copilot-enabled |
Copilot is enabled for the org | High |
branch-protection |
Default branch has protection rules | Medium |
code-scanning |
CodeQL or equivalent is configured | Medium |
dependabot-security |
Dependabot security alerts enabled | Medium |
secret-scanning |
Secret scanning with push protection | Medium |
org-instructions |
Org-level .github repo has instruction files |
High |
Auth requirements
This pillar requires a GitHub token with read:org and admin:org scopes. When the token is missing or insufficient, the pillar should gracefully skip with a clear message rather than failing the entire assessment.
Context
- Discussed in "AgentRC: Quick Chat on Futures" meeting
- Distinction made between what's assessable (repo configs, GitHub settings) vs. what's not (cultural practices, team workflows)
- Barton highlighted that org-level signals are critical for customer assessments
- This complements the maturity report card concept — org settings feed into the overall maturity tier
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing agentrc readiness command and repository-scoped readiness pillars to understand how optional assessments and skipped checks are represented. Map the proposed org-settings scope to the GitHub API signals and token requirements listed in the issue. Done means org and repository settings can be assessed when authorized, while missing or insufficient credentials produce a clear skip without failing the assessment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, typescript
- Domain
- cli, devops, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100