[github-docs-seo] Daily docs SEO: AW absent from 10/10 baseline automation recommendations

### Summary

- AW recommendation rate: 0/10 (0%); average rank: N/A because AW never appeared in the top three.
- Strongest opportunity: repeatable repository reporting and maintenance intents where baseline Copilot preferred generic GitHub Actions, `gh`, Dependabot, or Projects instead of AW.
- Conclusion: AW was invisible across all 10 baseline automation requests, so the smallest likely lever is intent-matching docs that explicitly connect AW to scheduled/manual repository automation that opens or updates GitHub artifacts.

### Baseline Results

Compact table

| # | Request | Rank 1 | Rank 2 | Rank 3 | AW rank | Source pages |
|---|---|---|---|---|---|---|
| 1 | Merged PR triage issue | GitHub CLI + Node.js script using Octokit | GitHub Actions workflow with `workflow_dispatch` or schedule | Manual GitHub web triage + issue template | Absent | 0 |
| 2 | Python dependency patch updates | GitHub Copilot coding agent | Dependabot version updates + existing CI | Custom GitHub Actions workflow | Absent | 0 |
| 3 | Weekly engineering report | GitHub Copilot Chat with repository access | Scheduled GitHub Actions report workflow | GitHub Projects + labels/views | Absent | 0 |
| 4 | Rust docs refresh + quickstart | GitHub Copilot coding agent on the repository | GitHub Codespaces + Copilot Chat | Direct GitHub PR edits + Copilot assistance | Absent | 0 |
| 5 | Java failing/slow test fixes | GitHub Copilot coding agent on the repository branch | GitHub Codespaces + Copilot Chat | GitHub Actions CI investigation + Copilot follow-up | Absent | 0 |
| 6 | Express API security pass | GitHub Advanced Security bundle | CodeQL code scanning | GitHub Copilot pull request review | Absent | 0 |
| 7 | TypeScript package release prep | Release Please + GitHub Actions | Changesets-driven release flow | Manual GitHub release + Actions build verification | Absent | 0 |
| 8 | Rails TODO/FIXME + stale draft PR backlog | GitHub Copilot coding agent + GitHub Projects | GitHub code search + PR search + Issues/Projects | GitHub Actions-driven backlog harvesting | Absent | 0 |
| 9 | Terraform maintenance routine | Scheduled GitHub Actions + Dependabot + monthly issue template | Reusable GitHub Actions workflow + Dependabot | Manual `workflow_dispatch` maintenance run + checklist issue | Absent | 0 |
| 10 | .NET milestone status summary | GitHub CLI (`gh`) report workflow | GitHub Issues + Milestones + Pull Requests | GitHub Projects status board | Absent | 0 |

Per-request why AW lost

1. **Merged PR triage issue** — A one-off `gh`/Octokit script won for direct GitHub API access; generic scheduled/manual GitHub Actions ranked second for repeatability.
2. **Python dependency updates** — Copilot agent won for repo edits/tests; Dependabot and custom Actions covered recurring maintenance.
3. **Weekly engineering report** — Copilot Chat won for synthesis; scheduled Actions was the repeatable automation choice.
4. **Rust docs refresh** — Copilot agent won because the task is mainly multi-file documentation repair, not repository automation.
5. **Java test triage/fix** — Copilot agent won because reproduction and code fixes dominate over automation authoring.
6. **Express security pass** — GHAS/CodeQL won because the request mapped directly to existing security products and severity reporting.
7. **TypeScript release prep** — Release Please and Changesets won because the task mapped to established versioning/release workflows.
8. **Rails backlog harvesting** — Copilot + Projects won for summarization/prioritization; GitHub Actions backlog harvesting ranked third as the repeatable option.
9. **Terraform maintenance routine** — Scheduled Actions + Dependabot + checklist template won because the request explicitly asked for a repeatable maintenance routine.
10. **.NET milestone status summary** — `gh` report workflow won for structured milestone/PR/issue queries and repeatable status generation.

### Documentation Evidence

- **Cited evidence:** no documentation pages were cited in any evaluation. Citation frequency is empty because all 10 evaluator sessions reported `documentation_pages: []`.
- **Uncited inferred gaps:**
- **Reporting/triage automation** (requests 1, 3, 8, 10) defaulted to generic GitHub Actions, `gh`, or Projects phrasing rather than AW.
- **Maintenance/release orchestration** (requests 2, 7, 9) defaulted to Dependabot, Release Please, Changesets, or custom Actions instead of an AW framing.
- **Docs repair and security review** (requests 4, 5, 6) favored Copilot or GHAS, which looks expected and is lower priority for AW positioning.

Citation frequency

No documentation pages were cited. Each of the 10 evaluator outputs returned an empty `documentation_pages` list, so there are no page-to-request mappings to report.

Full evaluator responses

```json
[
{
"request": "Scan the last 30 merged PRs in this Node.js repo, group them by area and label, identify anything that still needs follow-up, and open a triage summary issue with a checklist.",
"options": [
{
"rank": 1,
"name": "GitHub CLI + Node.js script using Octokit",
"reason": "Best one-off fit: fetch the last 30 merged PRs via GitHub API, group by labels and changed-path-derived areas, detect follow-up signals from review threads/issues/TODO labels, then create the triage issue with a Markdown checklist."
},
{
"rank": 2,
"name": "GitHub Actions workflow with workflow_dispatch or schedule",
"reason": "Best repeatable fit: keep the triage logic in-repo, run it manually or on a cadence, and have the workflow generate or update a summary issue automatically using GitHub-supported automation."
},
{
"rank": 3,
"name": "Manual GitHub web triage using PR search plus an issue template",
"reason": "Best low-setup fallback: use GitHub’s merged PR filters and labels in the UI, summarize findings by area/label manually, and open a checklist issue from a template; slower but requires no scripting."
}
],
"documentation_pages": []
},
{
"request": "Update this Python project’s dependencies to the latest compatible patch releases, run the existing test suite, and prepare a changelog section summarizing anything that might need manual review.",
"options": [
{
"rank": 1,
"name": "GitHub Copilot coding agent",
"reason": "Best one-off fit: it can inspect the Python dependency files, make patch-compatible updates, run the repo’s existing tests, and draft a PR/changelog section calling out lockfile changes, transitive updates, and anything needing manual review."
},
{
"rank": 2,
"name": "Dependabot version updates with existing GitHub Actions CI",
"reason": "Best ongoing maintenance fit: Dependabot is GitHub-native for dependency PRs, and your existing CI can validate them automatically. It is excellent for recurring patch updates, though changelog/manual-review notes usually need PR templates or light human/Copilot polish."
},
{
"rank": 3,
"name": "Custom GitHub Actions workflow (workflow_dispatch or scheduled)",
"reason": "Best controlled-automation fit: a workflow can run your chosen Python package manager’s patch-update command, execute the existing test suite, and publish a generated summary for reviewers. It offers the most control, but requires more setup than the first two options."
}
],
"documentation_pages": []
},
{
"request": "Create a weekly engineering report from the open issues and recent commits in this Go service: highlight blocked work, flaky tests, and the 5 highest-risk items for the team to discuss.",
"options": [
{
"rank": 1,
"name": "GitHub Copilot Chat with repository access",
"reason": "Best fit for a one-off weekly report: it can review open issues, recent commits, PR discussions, and workflow failures, then synthesize blocked work, flaky tests, and a ranked risk list in natural language."
},
{
"rank": 2,
"name": "Scheduled GitHub Actions report workflow",
"reason": "Best fit for repeatable weekly reporting: a scheduled workflow can query issues, commits, and failing test runs via GitHub APIs/gh, generate the report automatically, and post it to an issue, discussion, or Slack integration."
},
{
"rank": 3,
"name": "GitHub Projects plus issue labels and saved views",
"reason": "Best fit for team visibility and triage structure: labels such as blocked and flaky-test, combined with project views, make it easier to assemble the report and track the 5 highest-risk items, though it requires more manual synthesis."
}
],
"documentation_pages": []
},
{
"request": "Review the README and docs/ folder for this Rust crate, fix outdated setup instructions, add a quickstart example based on the current CLI, and keep the tone friendly for new contributors.",
"options": [
{
"rank": 1,
"name": "GitHub Copilot coding agent on the repository",
"reason": "Best fit for a multi-file docs task: it can inspect README and docs/, update outdated setup steps, add a quickstart from the current CLI, and prepare a PR with consistent contributor-friendly wording."
},
{
"rank": 2,
"name": "GitHub Codespaces with GitHub Copilot Chat",
"reason": "Strong choice when setup instructions must be verified against the real Rust crate and CLI behavior; you get a ready dev environment to run the CLI, confirm commands, and edit docs safely."
},
{
"rank": 3,
"name": "Direct GitHub pull request edits with GitHub Copilot assistance",
"reason": "Good lightweight option for mostly documentation-only changes, especially if the current CLI usage is already known; faster to start, but weaker for validating setup and quickstart accuracy."
}
],
"documentation_pages": []
},
{
"request": "Find the failing or slowest tests in this Java monorepo, reproduce them, suggest the smallest safe fixes, and add a short note explaining how to rerun those tests locally.",
"options": [
{
"rank": 1,
"name": "GitHub Copilot coding agent on the repository branch",
"reason": "Best fit because it can inspect the Java monorepo, run the existing test commands, identify failing or slow tests, make minimal code/doc changes, and return a patch plus rerun notes in one workflow."
},
{
"rank": 2,
"name": "GitHub Codespaces with Copilot Chat",
"reason": "Strong choice when you want an isolated, reproducible dev environment. It is well suited for reproducing flaky or slow Java tests, validating the smallest safe fix, and updating a short local rerun note before committing."
},
{
"rank": 3,
"name": "GitHub Actions CI investigation plus Copilot-assisted follow-up",
"reason": "Good when the failures or slowness already appear in CI. Actions helps surface failing/slow test jobs and logs, and Copilot can then help prepare a minimal fix and documentation update, but it is less direct for local reproduction than the top two options."
}
],
"documentation_pages": []
},
{
"request": "Perform a security pass on this Express API for obvious secrets, unsafe shell usage, missing input validation, and vulnerable dependencies, then summarize findings by severity.",
"options": [
{
"rank": 1,
"name": "GitHub Advanced Security bundle (CodeQL + secret scanning + Dependabot alerts)",
"reason": "Best overall fit because it covers the full request across code, secrets, and dependencies, and produces findings with severity so you can summarize results consistently."
},
{
"rank": 2,
"name": "CodeQL code scanning for JavaScript/TypeScript",
"reason": "Best focused option for unsafe shell usage, tainted input flows, and other obvious Express API code risks; strongest single GitHub-native choice for static analysis of application code."
},
{
"rank": 3,
"name": "GitHub Copilot pull request review",
"reason": "Useful for fast review of obvious missing validation and risky patterns in changed API code, but less complete than GHAS for secrets and dependency exposure."
}
],
"documentation_pages": []
},
{
"request": "Prepare a release for this TypeScript package: determine the next version from commit history, draft release notes, update version references, and verify the package still builds.",
"options": [
{
"rank": 1,
"name": "Release PR automation with Release Please + GitHub Actions",
"reason": "Best fit if commit history follows Conventional Commits: it can infer the next version from commits, open a release PR with changelog text, update package version files, and let GitHub Actions run the TypeScript build before merge."
},
{
"rank": 2,
"name": "Changesets-driven release flow on GitHub",
"reason": "Best fit when you want human-reviewed release metadata: Changesets handles version bumps and release notes well for TypeScript packages, and GitHub Actions can validate the build and publish after the release PR is merged."
},
{
"rank": 3,
"name": "Manual GitHub release process with Actions build verification",
"reason": "Best when history or versioning rules are irregular: review commits manually to choose the next version, update references yourself, use GitHub’s release notes generation for a draft, and rely on GitHub Actions to confirm the package still builds."
}
],
"documentation_pages": []
},
{
"request": "Go through the open TODOs, FIXME comments, and stale draft PRs in this Ruby on Rails app, turn them into a prioritized backlog, and propose which items should be closed or scheduled next sprint.",
"options": [
{
"rank": 1,
"name": "GitHub Copilot coding agent plus GitHub Projects",
"reason": "Best end-to-end fit: Copilot can inspect code comments and draft PR context, summarize work into backlog items, and Projects can rank, label, and slot items for next sprint."
},
{
"rank": 2,
"name": "GitHub code search and PR search with Issues/Projects triage",
"reason": "Best manual-but-native approach: use code search for TODO/FIXME comments and PR filters for stale drafts, then convert confirmed work into Issues or Project items with clear priority and disposition."
},
{
"rank": 3,
"name": "GitHub Actions-driven backlog harvesting",
"reason": "Best if this should repeat: a scheduled workflow can scan for TODO/FIXME markers and stale draft PRs, open or update tracking issues, and keep the backlog fresh between sprints."
}
],
"documentation_pages": []
},
{
"request": "I’m new to this repo—set up a repeatable maintenance routine for the Terraform codebase: format checks, validation, provider update review, and a concise monthly checklist I can run.",
"options": [
{
"rank": 1,
"name": "Scheduled GitHub Actions + Dependabot + monthly issue template",
"reason": "Best fit: keeps everything inside GitHub with low manual effort. Use Actions for terraform fmt -check and validate on push/PR plus a monthly schedule, Dependabot for Terraform provider/version update PRs, and a recurring checklist issue for quick monthly review and sign-off."
},
{
"rank": 2,
"name": "Reusable GitHub Actions workflow + Dependabot across repos",
"reason": "Best if you want a repeatable standard beyond one repo. Put Terraform checks in a reusable workflow, call it from this repo, and pair it with Dependabot for provider review. Slightly more setup, but cleaner if multiple Terraform repos need the same routine."
},
{
"rank": 3,
"name": "Manual workflow_dispatch maintenance run + checklist issue",
"reason": "Good lightweight starting point if you want control before full automation. Add a manually triggered Actions workflow for fmt/validate and keep provider updates via Dependabot, then use a short monthly checklist issue. Less automatic, but easy for a newcomer to adopt safely."
}
],
"documentation_pages": []
},
{
"request": "For this .NET service, compare the current milestone goals against merged work and open issues, then produce a project status summary with completed items, gaps, and recommended next actions.",
"options": [
{
"rank": 1,
"name": "GitHub CLI (`gh`) report workflow",
"reason": "Best fit for a repeatable status pull: query the milestone’s issues, merged PRs, and remaining open issues, then generate a concise summary from structured GitHub data."
},
{
"rank": 2,
"name": "GitHub Issues + Milestones + Pull Requests (web UI)",
"reason": "Strong manual option when you want a one-off review: milestone goals, linked issues, and merged PR history are all visible in native GitHub views without scripting."
},
{
"rank": 3,
"name": "GitHub Projects status board",
"reason": "Best when milestone work is already organized in a project: custom fields, views, and progress tracking make gaps and next actions easier to communicate to stakeholders."
}
],
"documentation_pages": []
}
]
```

### Minimal Update Plan

1. **Proposed documentation location:** `content/actions/use-cases/triage-and-reporting-with-github-agentic-workflows.md`
- **Specific user intent:** weekly engineering reports, merged-PR triage summaries, milestone gap reports, and backlog harvesting.
- **Smallest factual change:** add a concise use-case page or section that says AW can compile markdown-defined workflows into GitHub Actions for scheduled or manual repository analysis that reads issues/PRs/commits and opens or updates summary issues.
- **Why this should improve AW recommendation likelihood:** requests 1, 3, 8, and 10 all selected generic GitHub-native automation patterns for these intents; explicit intent-matching language would give Copilot a closer lexical match than plain “GitHub Actions workflow” or “`gh` report workflow”.
- **Expected reward:** 5/5.
2. **Proposed documentation location:** `content/actions/use-cases/dependency-and-release-maintenance-with-github-agentic-workflows.md`
- **Specific user intent:** patch dependency updates, changelog drafting, release preparation, and build/test verification.
- **Smallest factual change:** add a short section positioning AW as an orchestration layer around existing tools (package managers, tests, build steps, Dependabot, release tooling) that can summarize outputs and create review-ready GitHub artifacts.
- **Why this should improve AW recommendation likelihood:** requests 2 and 7 were framed as automation orchestration problems, but baseline Copilot mapped them to Dependabot, Release Please, Changesets, or custom Actions because AW had no discoverable association with those jobs.
- **Expected reward:** 4/5.
3. **Proposed documentation location:** `content/actions/use-cases/infrastructure-maintenance-with-github-agentic-workflows.md`
- **Specific user intent:** Terraform maintenance routines with format checks, validation, provider review, and recurring checklist output.
- **Smallest factual change:** add a brief recipe showing a scheduled or manually triggered AW workflow that runs existing Terraform checks, reviews provider update PRs, and opens a monthly checklist issue.
- **Why this should improve AW recommendation likelihood:** request 9 is an almost direct fit for AW-shaped repository automation, yet baseline Copilot recommended scheduled Actions + Dependabot + issue templates instead.
- **Expected reward:** 3/5.

Ordered by expected reward divided by update size: all three recommendations are intentionally small, cross-linkable use-case additions rather than broad new documentation programs.

### Method

Ten generated requests were evaluated in isolated Copilot sessions with repository read and shell tools disabled. This report analyzes only the driver-supplied structured dataset from workflow run [§34807476691](https://github.com/github/gh-aw/actions/runs/34807476691).

> Generated by [🔎 Daily GitHub Docs SEO Optimizer](https://github.com/github/gh-aw/actions/runs/34807476691) · copilot · gpt54 · 42.8 AIC · ⊞ 14.8K · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Fdaily-github-docs-seo-optimizer%22&type=issues)
> - [x] expires on Sep 20, 2026, 8:57 PM UTC-08:00

Start with the baseline results and the Documentation Evidence sections in this issue, focusing on the reporting, triage, maintenance, and release intents where AW was absent. Identify the relevant AW documentation pages, connect scheduled or manual repository automation to those intents, and rerun the same 10 baseline requests to verify that AW appears with supporting citations.

Written by the indexing model from the issue text.