githubnext / githubnext/gh-aw-cao

[self-care:docs-build-time-investigator] Collapse dashboard generation into the docs workflow run to remove cross-workflow wait time

Closed
#4,728 0 comments 0 reactions 0 assignees View on GitHub
self-care self-care:docs-build-time-investigator
Dominant language
JavaScript
Stars
3
Forks
1
Avg merge
49m
Merged PRs (30d)
837

Description

### Summary
The docs Pages workflow spends most of its time waiting for a separate dashboard-build workflow to run, even when dashboard data is already cached. Across 15 comparable successful scheduled `docs.yml` runs in the last 14 days, the full workflow median was 93s and p90 was 104.8s, while the in-workflow `build` job median was only 29s. The repeated overhead is the cross-workflow dashboard generation hop, not Astro or Pages packaging.

### Timing evidence
- Evidence window: 2026-09-07T11:22:22Z through 2026-09-07T15:33:19Z
- Comparable sample: 15 successful scheduled `docs.yml` runs
- Full workflow duration: median 93s, p90 104.8s
- Queue time: median 0s, p90 0s
- `dashboard` job: median 40s, p90 47.6s
- `build` job: median 29s, p90 36.4s
- `deploy` job: median 10s, p90 10.6s
- Nested `dashboard-build.yml` runs corresponding to the same docs window: `build` job median 18.5s, p90 25.1s; `activity` job median 5s, p90 6.1s
- Repeated step medians from trusted Actions metadata/logs:
- `docs.yml` → `dashboard` / `Dispatch dashboard build`: 34s median, 43s p90
- `docs.yml` → `build` / `Restore node_modules`: 6s median after cache hit
- `docs.yml` → `build` / `Build documentation`: 5s median
- `dashboard-build.yml` → `build` / `Restore collected activity data`: 4s median
- `dashboard-build.yml` → `build` / `Upload dashboard artifact`: 3s median
- Trusted cache evidence from logs:
- `actions/setup-node` npm cache restored successfully
- `actions/cache/restore` for `node_modules` hit consistently, but still restored ~457 MB (`Cache Size: ~436 MB`)
- `docs.yml` dashboard cache misses because the key is the dispatched run id/attempt (`cao-dashboard--`), so `Mount dashboard at /cao` falls back to artifact download rather than a reusable cache hit

### Bottleneck
`docs.yml` always dispatches `.github/workflows/dashboard-build.yml` and then waits for that separate workflow to finish before the Pages build can continue. On the representative run `34132817395`, the outer workflow spent ~45s in `Dispatch dashboard build`, while the nested dashboard build itself took ~40s. By contrast, Astro finished in about 3.5s and Pages artifact upload in about 3s. This means the dominant cost is orchestration and cross-workflow artifact handoff for dashboard generation, not documentation compilation.

### Recommended changes
Replace the cross-workflow dashboard handoff in `.github/workflows/docs.yml` with an in-workflow reusable path that generates or restores the dashboard artifact inside the same workflow run.

Concretely:
- Remove the separate `dashboard` job dispatch from `.github/workflows/docs.yml` and instead call the dashboard build logic directly in the same workflow, either by:
- moving the dashboard generation steps into a local job in `docs.yml`, or
- refactoring `.github/workflows/dashboard-build.yml` into a reusable workflow (`workflow_call`) whose build job runs within the same run context.
- Preserve the existing cached-activity behavior from `.github/workflows/dashboard-build.yml`, but publish the dashboard output directly to the downstream Pages build job through same-run job outputs/artifacts instead of a second workflow run plus lookup by external run id.
- Keep the existing Node/npm caching, because dependency installation is not the current bottleneck.

Relevant source locations:
- `.github/workflows/docs.yml`: `dashboard` job and `build` step `Restore dashboard from cache`
- `.github/workflows/dashboard-build.yml`: `activity` and `build` jobs, especially `Restore collected activity data`, `Assemble Dashboard Language site`, and `Upload dashboard artifact`

### Expected effect and validation
Expected effect: remove roughly the 34s median dispatch/wait cost and most cross-workflow artifact lookup overhead, which should save about 30-40s per scheduled docs deployment. That is about 32%-43% of the current 93s median full workflow time, exceeding the 15% / 60s action threshold on relative impact and approaching it in absolute time.

Validation plan:
1. Compare at least five scheduled runs before and after the change.
2. Recompute full-workflow median and p90 plus `build` and `deploy` job medians.
3. Confirm dashboard freshness and correctness by checking the resulting `cao/sources.json` and deployed dashboard artifact contents.
4. Confirm no regression in Pages deployment success rate.

Confidence: medium-high. The bottleneck is stable across 15 comparable scheduled runs and is directly visible in trusted run metadata and logs.

### Caveats
- This recommendation is about eliminating orchestration overhead, not weakening freshness. The dashboard data refresh decision should continue to honor the current `mode` behavior.
- If the separate workflow exists for isolation or permissions reasons, preserve those guarantees when refactoring to a reusable same-run implementation.
- Open-issue deduplication could not be verified because GitHub issue listing returned transient 502 errors in this worker environment.

### Control Plane
- Correlation ID: `34138626350-351`
- Central repository: `githubnext/gh-aw-cao`
- Control plane run: https://github.com/githubnext/gh-aw-cao/actions/runs/34138626350

### References
- Workflow source: `.github/workflows/docs.yml`
- Workflow source: `.github/workflows/dashboard-build.yml`
- Representative docs run: https://github.com/githubnext/gh-aw-cao/actions/runs/34132817395
- Representative nested dashboard build run: https://github.com/githubnext/gh-aw-cao/actions/runs/34132830208
- Additional comparable docs runs: `34116275934`, `34117084784`, `34118291550`, `34119694874`, `34122630825`, `34124232444`, `34125121899`, `34127306145`, `34128118507`, `34132288563`, `34133588592`, `34134743209`, `34136130112`, `34137984246`, `34138768990`

> Generated by [SelfCare / Docs Build Time](https://github.com/githubnext/gh-aw-cao/actions/runs/34138994774) · pi · gpt54 · 47.2 AIC · ⌖ 8.5 AIC · ⊞ 8.8K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-cao+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-cao%2Fself-care-docs-build-time-investigator%22&type=issues)
> - [x] expires on Sep 21, 2026, 3:41 PM UTC

Contributor guide

Open the contributing guide

Research direction

Read .github/workflows/docs.yml and .github/workflows/dashboard-build.yml, starting with the docs dashboard job, the Restore dashboard from cache step, and the dashboard activity/build jobs. Trace how dashboard data and artifacts move between runs, then define the same-run workflow path while preserving mode behavior and permissions. Done means scheduled runs avoid the cross-workflow wait, dashboard contents remain fresh, and at least five before/after runs show improved timing without Pages failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
build-system, ci-cd
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.