githubnext / githubnext/gh-aw-wizard

[aw] Prompt Optimization reported incomplete result

Open
#268 1 comment 0 reactions 0 assignees View on GitHub
agentic-workflows
Dominant language
JavaScript
Stars
3
Forks
0
Avg merge
3h 8m
Merged PRs (30d)
244

Description

### Workflow Failure

**Workflow:** [Prompt Optimization](https://github.com/githubnext/gh-aw-wizard/blob/main/.github/workflows/prompt-optimization.md)
**Branch:** main
**Run:** https://github.com/githubnext/gh-aw-wizard/actions/runs/34162766948

> [!WARNING]
> **Task Could Not Be Completed**: The agent reported that the task could not be performed due to an infrastructure or tool failure.

**Reasons:**
- The prompt claims Ollama cache was restored and GGUF proxies for both models were downloaded, but this was not actually done in this run.

Checks performed:
- No open PR titled "Prompt optimization:" exists (confirmed via `gh pr list`), so the existing-PR guard did not apply.
- Searched the entire filesystem for `*.gguf` files, Ollama model blobs, and Hugging Face cache directories: none found. `/home/runner/.ollama` did not exist; a stray `.ollama` dir under a chroot-home tmp path was also empty (0 models).
- No Ollama server was listening on 127.0.0.1:11434 at task start (connection refused).
- Manually started `ollama serve` (binary present at /opt/hostedtoolcache/ollama/0.33.2/x64/ollama) — it started but logged that its `llama-server` binbinary is missing ("Run 'cmake -S llama/server --preset cpu && cmake --build --preset cpu' first"), meaning it cannot actually run inference even if a model were loaded.
- Attempted `ollama pull hf.co/unsloth/SmolLM2-360M-Instruct-GGUF:Q4_K_M` to fetch the required proxy model: failed with "Forbidden" — outbound network access to huggingface.co is blocked by the sandbox's egress proxy.

Net result: neither the eval model (Qwen2.5-1.5B) nor the iOS eval model (SmolLM2-360M) could be obtained or served, so no `--evaluate`/`--score` command from scripts/prompt-optimizer.mjs could produce a real measurement. Running the hill-climbing loop without a working model would only fabricate scores, which the task explicitly forbids ("Never claim an improvement that the harness did not measure."). No repository files were changed; no PR, review, or review comment was created.

This is a structured incompletion signal (`report_incomplete`), not a real task outcome. Any other safe outputs emitted alongside this signal (e.g., comments) describe the failure state, not a completed review or action.

### Action Required

**Assign this issue to an agent** to debug and fix the issue.

Debug with any coding agent

Use this prompt with any coding agent (GitHub Copilot, Claude, Gemini, etc.):

````
Debug the agentic workflow failure using https://raw.githubusercontent.com/github/gh-aw/main/debug.md

The failed workflow run is at https://github.com/githubnext/gh-aw-wizard/actions/runs/34162766948
````

Manually invoke the agent

Debug this workflow failure using your favorite Agent CLI and the `agentic-workflows` prompt.

- Start your agent
- Load the `agentic-workflows` skill from `.github/skills/agentic-workflows/SKILL.md` or
- Type `debug the agentic workflow prompt-optimization failure in https://github.com/githubnext/gh-aw-wizard/actions/runs/34162766948`

> [!TIP]
>
> Stop reporting this workflow as a failure
>
> To stop a workflow from creating failure issues, set `report-failure-as-issue: false` in its frontmatter:
> ```yaml
> safe-outputs:
> report-failure-as-issue: false
> ```
>
>

> Generated from [Prompt Optimization](https://github.com/githubnext/gh-aw-wizard/actions/runs/34162766948) · copilot · 29.5 AIC · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-wizard+is%3Aissue+%22gh-aw-workflow-id%3A+prompt-optimization%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with .github/workflows/prompt-optimization.md and scripts/prompt-optimizer.mjs, then review the agentic-workflows skill referenced in the issue. Reproduce the run's model setup and serving failures, checking the Ollama binary and blocked model download path. Done means both evaluation models are available and --evaluate/--score produces real measurements without fabricated scores.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, github-actions, javascript, ollama
Domain
ai, ci-cd, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.