github / github/gh-aw

Pydantic AI as an agentic engine

Open
#50,971 12 comments 1 reaction 2 assignees Claimed by @pelikhan View on GitHub
ai-inspected community
Dominant language
Go
Stars
5.1k
Forks
541
Avg merge
5h 46m
Merged PRs (30d)
760

Description

hey guys! @strawgate brought agentic workflows to our attention a couple months ago and we've been
using them since then (https://github.com/pydantic/pydantic-ai/tree/main/.github/workflows, https://github.com/pydantic/pydantic-ai/issues/7211, https://github.com/pydantic/pydantic-ai/pull/7253#pullrequestreview-4878979269).

in parallel we've been running our own handrolled agent that does triage and review in our repo https://github.com/pydantic/pydantic-ai/issues/7235#issuecomment-5209033345.

we like agentic workflows and want to keep using them, while also dogfooding our own stack, which
is why we want to build a Pydantic AI based engine https://github.com/pydantic/pydantic-ai-harness/pull/569 following the third-party model in your https://github.github.io/gh-aw/reference/engines/

claude's helping me build it so I'll leave its description below, + some questions we have for you guys so we can make the right choices

thank you for your time and attention!

---

**the shape.** An owner-maintained engine definition (`engine: {id: pydantic-ai, behaviors: ...}`)
published in [pydantic-ai-harness](https://github.com/pydantic/pydantic-ai-harness) and imported
pinned to a tag, backed by a headless coding-agent CLI installed via `runtimes: {uv: {}}` +
`pre-agent-steps` (the `aider.md` pattern), with `secret-strategy: universal-llm-consumer` for
auth. Nothing needs to live in your repo or binary -- users import the definition from ours.

**built already** ([pydantic-ai-harness#569](https://github.com/pydantic/pydantic-ai-harness/pull/569)):
the engine definition and a smoke workflow, compile-tested:

- a new `pydantic-ai` engine id imported from a shared definition compiles clean with `--strict` on
v0.85.4. we bisected the releases: v0.84.3 still rejects new ids, the #50145 fix lands in v0.84.4.
we'll document v0.85.4 as our minimum supported version.
- cross-repo pinned imports (`owner/repo/path.md@tag`) resolve, SHA-pin, and vendor correctly --
tag, branch, and bare-SHA refs all work.
- one docs/UX note from testing: the bare-string form (`engine: pydantic-ai`) errors even when the
definition is imported -- only the object form (`engine: {id: pydantic-ai}`) works, and the error
message doesn't hint at that. cost us a minute; will cost every new user a minute.

**being built now**: the CLI itself. pydantic-ai-harness ships all the pieces a coding agent needs
(sandboxed filesystem + shell tools, repo context loading, planning, context compaction) but no
console entry point yet -- that's the gap we're closing, designed against your engine contract from
day one: prompt in, one-shot run, exit code out, MCP config consumed from the gateway so safe
outputs flow through the standard `safeoutputs` server.

**three questions where your answers change what we build:**

1. **install.** `behaviors.installation.package-manager` values other than `npm` compile clean but
silently emit no install step. we're fine on the `pre-agent-steps` path -- but a compile warning
for non-npm values would save the next Python/Rust engine author a confusing afternoon. want an
issue/PR for that?
2. **logs and metrics.** behavior-defined engines get no log parser: no conversation rendering in
the step summary, and `gh aw logs` / `gh aw audit` report zero tokens and turns (AI-credit
accounting through the proxy still works). we get why -- #20416 scoped declarative log parsing
out as a non-goal -- but that was before imported engines became the recommended route for third
parties. is there appetite for a hook here, e.g. a documented streaming-JSONL schema plus
`parse_custom_log.cjs` promoted to a real contract? we'd emit whatever schema you standardize,
and we're happy to contribute the parser.
3. **discoverability.** once our definition is live and smoke-tested, would you take a link to it
from the [third-party agent guide](https://github.github.io/gh-aw/guides/third-party-agent/)?
not asking for a row in the engines table or any support commitment from you -- just a pointer
for people searching "how do I run X on gh-aw".

whatever shape this takes, we intend to maintain it long-term -- so if any assumption above looks
wrong, tell us now and we'll adjust before we ship :)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.