github / github/gh-aw

Support `strategy.matrix` on the `agent` job for parallel AI execution

Open
#26,598 1 comment 0 reactions 0 assignees View on GitHub
community
Dominant language
Go
Stars
5.1k
Forks
541
Avg merge
5h 46m
Merged PRs (30d)
760

Description

## Problem

Currently, the `strategy` / `matrix` support in gh-aw only applies to **custom jobs** defined under `jobs:` in the frontmatter (e.g., the [BatchOps pattern](https://github.com/github/gh-aw/blob/main/docs/src/content/docs/patterns/batch-ops.md)). The **agent job itself** — the core AI execution job — does not support matrix strategy, meaning it always runs as a single instance per workflow run.

This is a limitation for use cases like **randomized testing at scale**, where you want to dispatch N parallel agent threads with different parameters (random seeds, test configs, shards) and have each agent instance work independently and concurrently.

## Use Case

I want to run **randomized test campaigns** where:

1. Multiple agent instances run **in parallel** as matrix jobs (e.g., 5–10 concurrent threads)
2. Each instance receives different matrix parameters (e.g., `seed`, `test-suite`, `scenario`)
3. Each agent independently generates and executes randomized test scenarios using its own AI session
4. Results from all parallel agents are aggregated at the end

Today, achieving this requires either:
- Running agents sequentially (slow, defeats the purpose)
- Using `workflow_dispatch` fan-out with multiple workflow runs (wasteful, hard to coordinate, hits per-engine concurrency)
- Custom jobs with matrix that call back into agent workflows (fragile, not a first-class pattern)

## Proposed Solution

Allow `strategy` on the agent job, either at the top level or under `engine`:

```yaml
---
on:
workflow_dispatch:
inputs:
threads:
default: "5"

strategy:
matrix:
thread: [0, 1, 2, 3, 4]
fail-fast: false
max-parallel: 5

engine:
id: copilot
concurrency:
group: "gh-aw-copilot-${{ github.workflow }}-${{ matrix.thread }}"

tools:
bash: ["pytest", "jq"]
---

# Randomized Test Runner — Thread ${{ matrix.thread }}

Run randomized test suite using seed derived from thread index ${{ matrix.thread }}.
Each thread independently selects and executes a random subset of tests.
```

Key requirements:
- **Matrix parameters available in markdown body** via `${{ matrix.* }}` expressions
- **Per-instance concurrency groups** so parallel agent instances don't cancel each other
- **`fail-fast: false`** support so one failing agent doesn't kill the others
- **`max-parallel`** to cap concurrent AI sessions and manage token/API costs

## Prior Art

- Issue #19402 removed the unusable `strategy` schema from custom jobs (was an empty-object-only definition). The BatchOps pattern now shows working matrix on custom jobs, but not on the agent job.
- Issue #18101 addressed per-engine concurrency blocking parallel runs.
- The `concurrency.job-discriminator` feature already solves part of this (fan-out without cancellation), but doesn't provide matrix parameters or parallel agent instances within a single workflow run.

## Alternatives Considered

| Approach | Limitation |
|----------|-----------|
| `workflow_dispatch` fan-out (dispatch N separate runs) | No shared context, hard to aggregate, per-engine concurrency blocks |
| `job-discriminator` with dispatch-ops | No matrix parameters in the agent prompt |
| Custom job with matrix + `assign-to-agent` safe output | Fragile, adds a full extra job layer, not first-class |

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.