github / github/gh-aw

[agentic-token-optimizer] Optimize AIC usage in Deep Report workflow (dedup batching, working-set reuse, incremental AgentDB ingestion)

Open
#61,158 0 comments 0 reactions 0 assignees View on GitHub
optimization token-cost
Dominant language
Go
Stars
5.1k
Forks
541
Avg merge
5h 48m
Merged PRs (30d)
773

Description

### Target Workflow: `Deep Report` (`.github/workflows/deep-report.md`)

**Why selected**: Highest total AIC (279.47) among top-10 workflows this week, has not been optimized in the last 14 days (no entry in `optimization-log.json`), and its name does not contain "Token" (so it is not a self-target of the audit/optimizer tooling).

### Analysis Period & Runs Analyzed

- Window: last 7 days (2026-09-08 → 2026-09-15)
- Runs found via `gh api actions/workflows/deep-report.lock.yml/runs`: **31** (27 success, 4 failure)
- Deep MCP-tool-call trace available for 1 representative run: [§34969464054](https://github.com/github/gh-aw/actions/runs/34969464054) (run #345, 2026-09-15, success, AIC 279.47, 80,823 tokens, 22 action-minutes, 6 safe outputs: 5 issues + 1 discussion)
- Daily-snapshot AIC trend (from repo-memory): 2026-09-08 → 154.76, 2026-09-10 → 343.91, 2026-09-11 → 305.49, 2026-09-14 → 0 (activation failure, no agent run), 2026-09-15 → 248.05
- All 4 failed runs in the window failed at the **`activation`** job (infra/preflight), not agent logic — see caveats.

### Cost Profile

| Metric | Value |
|---|---|
| Total AIC analyzed (5 sampled days) | 1,052.21 |
| Avg AIC / completed run | ~263.05 |
| Raw tokens (sample run) | 80,823 |
| Avg action-minutes / run | ~20 |
| Working-set rebuild factor (sample run) | 1.447 (44.7% context re-fetch overhead) |
| Cumulative vs peak input tokens (sample run) | 41,751 cumulative vs 28,845 peak → 12,906 "excess" tokens across 42 invocations |
| MCP tool calls in sample run | 92 (60 of them `github.issue_read`) |

### Ranked Recommendations

#### 1. Batch the per-candidate issue dedup lookups (est. ~10% AIC/run savings)

**Evidence**: In the sample run, the agent made **60 individual `github.issue_read` calls** (71 bytes input each) inside the 21-minute window, versus only 4 `search_issues` calls. The workflow's "Dedup gate" instruction ("Search open issues for similar work using title keywords...") is being executed as one `issue_read` per candidate/keyword instead of a batched search.

**Action**: Rewrite the dedup-gate instructions in Step 2.7 and the "Actionable Task Creation" section to require a **single `search_issues` query per candidate task** (using combined keyword/label filters) instead of iterative `issue_read` calls, and explicitly forbid reading full issue bodies one-by-one unless a candidate match is already shortlisted by search.

#### 2. Reduce working-set rebuild overhead (est. ~8% AIC/run savings)

**Evidence**: `rebuild_factor: 1.447` with `rebuild_excess_tokens: 12,906` across 42 invocations indicates the agent is re-fetching/re-reading previously-seen context (discussions.json, issues.json, repo-memory files) multiple times rather than caching it in-session.

**Action**: Add an explicit instruction near the top of the prompt (Step 0/Step 1) to load `discussions.json`, `issues.json`, and repo-memory files **once** into working memory and explicitly reference already-loaded content in later steps instead of re-reading files that were already loaded earlier in the same run.

#### 3. Make AgentDB ingestion incremental (est. ~10% AIC/run savings)

**Evidence**: Step 1 instructs "Ingest the filtered discussion data into AgentDB memory" every run, re-ingesting the full 7-day discussion corpus even though Step 0 already checks `last_analysis_timestamp.md` and repo-memory tracks `processed-discussions.json` for the separate task-mining step.

**Action**: Extend the "less than 20 hours ago" freshness check from Step 0 to gate the AgentDB ingestion step too — only re-ingest discussions newer than `last_analysis_timestamp`, not the full 7-day window, on every 6-hour scheduled run.

### Structural Optimization: Inline Sub-Agent for Code-Quality Task Mining

The workflow already has one inline sub-agent (`issues-analyst`, small model), so a new one is only recommended because Step 2.7 ("Mine Discussions for Code Quality Tasks") is clearly separate, independent, and extractive.

**Candidate**: `task-miner` sub-agent
- **Task**: For each unprocessed discussion (filtered against `processed-discussions.json`), extract 0-N candidate code-quality tasks meeting the Specific/Actionable/Valuable/Scoped/Independent criteria already defined in Step 2.7, and output a structured JSON list of candidates with title, rationale, and source discussion link.
- **Why a smaller model fits**: The work is per-discussion classification/extraction against a fixed rubric — no cross-source synthesis required.
- **Score**: Independence 3/3 (only needs `discussions.json` + `processed-discussions.json`), Small-model adequacy 3/3 (rubric-based extraction), Parallelism 2/2 (can run alongside `issues-analyst` and the main discussion-intelligence pass), Size 2/2 (loops over multiple discussions) → **10/10, strong candidate**.
- **Invocation change**: Replace the inline Step 2.7 instructions in the main prompt with a call to `task-miner`, passing the filtered discussion set, and have the main agent only perform the final dedup-against-open-issues pass and issue creation using `task-miner`'s output.

**Estimated combined savings**: ~55-65 AIC/run (~20-25%) after accounting for overlap between the above changes.

### Caveats

- Sample size for detailed MCP tool-call tracing is 1 run; the dedup-gate and rebuild-factor findings should be validated against a second run before implementation.
- All 4 failed runs in the window failed at the `activation` job (GitHub Actions infra/preflight), not agent logic — no reliability optimization is recommended for these failures.
- `turns: 0` is reported for all sampled runs (Claude engine may not populate this field); turn-count-based analysis was not possible.

> Generated by [Agentic Workflow AIC Usage Optimizer](https://github.com/github/gh-aw/actions/runs/34984627441) · copilot · auto · 72.2 AIC · ⊞ 10.8K · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Fagentic-token-optimizer%22&type=issues)
> - [x] expires on Sep 22, 2026, 7:04 AM UTC-08:00

Contributor guide

Open the contributing guide

Research direction

Start by reading .github/workflows/deep-report.md, especially Steps 0, 1, and 2.7, then compare its behavior with the cited representative run and optimization data. Done means the dedup searches are batched, previously loaded context is reused, AgentDB ingestion is incremental, and the proposed task-miner integration is evaluated against a second run.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, devops
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.