githubnext / githubnext/gh-aw-wizard

[deep-research] Deep Research: gh-aw pattern library analysis (2026-08-31 snapshot)

Open
#279 0 comments 0 reactions 0 assignees View on GitHub
deep-research
Dominant language
JavaScript
Stars
3
Forks
0
Avg merge
3h 8m
Merged PRs (30d)
244

Description

## Deep Research: gh-aw Pattern Library Analysis

**Source**: `patterns/manifest.json` (generated 2026-08-31T09:42:11Z from 223 source repos, 175 active workflows, 671 total workflows scanned) plus 29 archetype files under `patterns/archetypes/`.

### Archetype coverage
- Manifest lists **29 archetypes** with empirical run data ranging from `count: 0` (curated, unmeasured) to `count: 356` (`custom`, the largest bucket).
- Archetypes with measured success (n>0), ranked:
- `daily-test-improver` 100% (n=3)
- `documentation-updater` 68% (n=9)
- `issue-triage` 52% (n=72)
- `custom` 50% (n=356, aggregate bucket)
- `dependency-monitor` 50% (n=48)
- `code-improvement` 46% (n=73)
- `pr-review` 42% (n=63)
- `status-report` 38% (n=36)
- `repo-maintainer` 33% (n=8)
- `content-moderation` 0% (n=3)
- 20 archetypes (e.g. `accessibility-expert`, `security-scanner`, `linter-workflows`, `community-digest`, `pr-iteration-loop`, etc.) have `count: 0` — curated/no empirical runs yet, but each still carries a recommended-trigger profile, prompt-style guidance, and tips derived from design heuristics rather than measured outcomes.

### Recommended configuration defaults (`config_defaults`)
- No pinned model by default (`model: null`).
- Timeout by trigger: `schedule` 30 min; `issues`, `push`, `slash_command`, `workflow_run`, `discussion`, `pull_request` 15 min each (archetypes may override upward, e.g. `pr-iteration-loop` at 45 min).
- Prompt size sweet spot: 3,000–8,000 bytes; phase-based archetypes (code-improvement, daily-test-improver, repo-maintainer, pr-iteration-loop, linter-workflows, security-scanner) recommend larger 5,000–20,000 byte prompts.

### Trigger combinations (`trigger_combos`, 15 tracked)
- 13 of 15 combos are 90–100% success and marked "Recommended"; none are flagged as risky in the curated list.
- Standouts: `bots+check_suite+permissions+steps`, `reaction+schedule+stop-after`, `pull_request_target+roles+skip-bots`, `issues+roles+skip-if-no-match` — all 100% (n=10 each).
- Lone `reaction` trigger: 99% success (n=90), the largest sample size in the table.
- Softest tracked combo: `bots+roles+schedule+stale-check` at 90% (n=20).

### Configuration profiles (archetype × trigger × safe-output detail)
Notable contrasts within the same archetype:
- `code-improvement`: schedule+skip-if-match → create-pull-request measured **0%** (n=23); workflow_run → add-comment/create-issue also **0%** (n=20). Confirms workflow_run chaining risk.
- `custom`: schedule → create-pull-request+noop is the standout profile at **95.2%** (n=21, confidence 0.773), followed by schedule → create-issue+noop+threat-detection at 83.9% (n=31) and schedule → create-issue+noop at 80.0% (n=20).
- `dependency-monitor`: schedule → create-pull-request 37.5% (n=56); adding `allowed-domains` dropped to 31.0% (n=42).
- `issue-triage`: issues+roles → add-comment+add-labels+assign-to-agent underperformed at 12.2% (n=82) — assignment step correlates with lower success.
- `status-report`: schedule → create-issue 18.0% (n=61); adding mentions/allowed-github-references → **0%** (n=40).

### Anti-patterns (20 entries, all 0% success, each seen once)
All are single-repo, single-occurrence named patterns (e.g. `daily-repo-status` ×2, `supply-chain-review` ×2, `ci-coach`, `code-simplifier`, `arm-api-review`, `hero-scenarios`, `data-plane-api-review`, `dash`, `tester`, `scribe`, `archie`, `sentinel`, `sdk-build-repair`, `ci-doctor`, `news-translate`, `localize-pt`, `issue-go-yes`, `issue-triage`). Common theme: broad, unscoped daily/status/supply-chain/CI-coach mandates without narrow targets.

### Research findings
- Outcomes are bimodal: 38% of workflows always succeed, 21% always fail, 41% mixed — averages are misleading.
- Explicit DO NOT constraints correlate with 61% higher health likelihood (p=0.009).
- `workflow_run` chaining: 13–16% success — prefer pre-steps or schedule.
- Pre-steps correlate with +13pp internal / +5pp community activity.
- Active workflows have 35–48% larger prompts than inactive ones.
- 32% of workflows are unmodified template clones, with lower success than customized ones.

### Archetype-level tips worth highlighting
- `status-report`, `documentation-updater`, `code-improvement`: use `skip-if-match`/`expires`/`close-older-issues` to avoid duplicate scheduled outputs.
- `dependency-monitor`: explicitly list monitored ecosystems; add package-registry domains to `network.allowed` since "defaults"/"github" alone don't cover them.
- `pr-review`: set `tools.github.min-integrity` to restrict acting on untrusted content.
- `security-scanner` (0 runs, curated): scope to a bounded recent commit window, report as code-scanning alerts (not issues), call `noop` when clean.
- `pr-iteration-loop` (0 runs, curated): persist iteration history in repo-memory, accept iterations only when they pass verification, pause after bounded retries.

No `degraded_workflows` were flagged in this manifest snapshot.

---
*This is a read-only analysis of the committed pattern library only; no scanner was run and no other repository files were consulted.*

> Generated by [🔬 Deep Research](https://github.com/githubnext/gh-aw-wizard/actions/runs/34745288870) · copilot · auto · 39.9 AIC · ⌖ 4.87 AIC · ⊞ 6.2K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-wizard+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-wizard%2Fdeep-research%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with patterns/manifest.json and compare the 29 files under patterns/archetypes/. Treat the supplied metrics and research findings as the current baseline; the issue does not define a requested change or acceptance criteria, so clarify the intended deliverable before modifying anything.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.