Design: Project workflow engine — transition event stream, node-level state inspection, and graph export

# Design: Project workflow engine — transition event stream, node-level state inspection, and graph export

**Status:** open for discussion
**Scope:** `agentteams-controller` + `plugins/teamharness` (no worker runtime changes, no new infrastructure)

## Background

The project workflow API (`GET /api/v1/projects/{id}/workflow`) already returns a snapshot aligned with the LangGraph `StateSnapshot` model — `nodes / edges / next / interrupts / values` (see the source attribution in `agentteams-controller/internal/server/project_handler.go`). The DAG is authored at runtime by the team leader (`plan_dag`) and can be re-planned several times over a project's life. That dynamism is a feature, but it has a consequence: **when a transition happens, nothing records it.**

Concrete gaps today:

1. **No transition history.** Task status is persisted only as its current value in project/task metadata (object storage). If a task went planned → assigned → in_progress → submitted → revision, only `revision` survives. Debugging "why did the workflow stall here?" currently means mining Matrix room history.
2. **No node-level inspection.** The workflow snapshot is a flat list; there is no per-task endpoint aggregating state + spec + deliverables + history. (`?includeTasks=true` exists but returns a flat current-value list.)
3. **No graph export.** UIs must parse the JSON snapshot and draw their own graphs. A first-party mermaid export would let dashboards, docs, and chat replies render the workflow with one line.
4. **Partial notifications.** A code-level completion notification for task submission is in flight (see #1206). There is no uniform, queryable record of *all* transitions.

## Proposed design

Four parts, grouped into two PRs. All state stays in object storage; PR-1 adds no writes at all.

### PR-1 — read-only observability (controller)

**1. Mermaid export.** Extend the workflow endpoint with a graph export (e.g. `GET /api/v1/projects/{id}/workflow?format=mermaid`) rendering the existing snapshot as a mermaid `flowchart` with node status classes (pending / active / submitted / terminal / interrupted). Pure rendering of data already computed by `buildWorkflow` — no new state, no behavior change.

**2. Node-level state inspection.** A new per-task endpoint (e.g. `GET /api/v1/projects/{id}/tasks/{taskId}`) aggregating:

- current task state, assignee, dependencies (from project meta)
- the task spec/brief (from the task directory)
- deliverables list (paths, not contents)
- append-only transition `history[]` (populated by PR-2; an empty array until then)
- a trace reference: worker-side tracing already tags entry spans with `agentteams.task.id` / `agentteams.project.id` (`plugins/teamharness/adapters/qwenpaw/task_trace.py`), so a UI can deep-link from a node into a tracing backend.

### PR-2 — engine discipline (teamharness taskflow)

**3. Explicit transition table.** A single source of truth for task-state transitions — a candidate sketch (to be validated against every transition call site before implementation):

```
planned -> assigned, cancelled
assigned -> in_progress, blocked, cancelled
in_progress -> submitted, blocked, cancelled
submitted -> completed, revision, blocked
completed / revision / blocked / cancelled -> (terminal, per TERMINAL_TASK_STATUSES)
```

enforced at the transition choke points; an invalid transition is rejected with a structured error instead of being silently accepted. The table should also document the semantics of each state (e.g. `revision` being terminal until re-planned).

**4. Append-only node history.** Each accepted transition appends `{ts, from, to, actor, action, note?}` to the task's history, persisted alongside task state. Size/retention policy to be agreed (Q3).

**5. Transition event stream.** Each accepted transition emits a structured event `{project_id, task_id, from, to, actor, ts, action, note?}` — persisted and listable with a cursor (audit + dashboards), and driving room notifications (a superset of #1206's completion notification; see Q4).

**6. Worker progress self-report.** An optional lightweight action for the assigned worker to attach a one-line progress note to the task while working (appended to history, surfaced in the node panel). It causes no state change.

## Boundaries and non-goals

- Object storage remains the source of truth; PR-1 adds no writes, PR-2 only appends.
- **No graph execution engine, no checkpointing / time travel.** The DAG is dynamic by design (LLM-authored, re-planned), so replay semantics do not apply. History is for audit and UX, not for rolling state back.
- **No unification with agent-internal TODO models** — different layers (session-scoped working memory vs project-scoped shared entities).
- **No dependency-edge synchronization** between layers; `depends_on` in project meta remains the single cross-session truth.
- Human-intervention endpoints (pause / resume / report) keep their current behavior; they simply become additional transition actors covered by the same table.

## Relationship to in-flight work

- **#1206 (task completion notification):** PR-2's event stream is a superset of it. Proposal: land #1206 first; PR-2 then routes notifications through the event stream (or retires the special case). See Q4.
- **#1183 (durable task continuation state):** touches the same project metadata and `project_handler.go`. PR-2's history persistence must coexist with its continuation records — the storage layout should be agreed between the two PRs before either lands.

## Open questions

1. **Mermaid endpoint shape:** `?format=mermaid` on the existing route, or a dedicated subresource (`/workflow/graph`)? What does house style prefer?
2. **Event delivery:** a persisted cursor-list endpoint is the minimum. Should the controller also expose SSE (or webhooks) for live dashboards, or is polling enough for v1?
3. **History / event storage and caps:** append to per-task state files vs a project-level event log? Any retention or size policy precedent in the repo?
4. **Coordination with #1206:** should the completion notification stay a special case, or become the first consumer of the event stream?
5. **Enforcement point:** today, transitions are written from two places — the teamharness MCP path (agent-driven) and the controller (human intervention) — both writing the same metadata. Should the transition table live in one shared location enforced by both, or should the agent path be funneled through a controller API for a single enforcement point?
6. **API versioning:** should the new endpoints be additive under `/api/v1/...` or version-gated?

## Plan

This issue is a design record. If the direction is accepted, the PRs would be:

- **PR-1:** mermaid export + per-task inspection endpoint (controller, read-only)
- **PR-2:** transition table + node history + event stream + progress self-report (teamharness, additive)

---

# 设计：项目工作流引擎 —— 转换事件流、节点级状态检视与图导出

**状态：** 开放讨论
**范围：** `agentteams-controller` + `plugins/teamharness`（不改 worker runtime，不引入新基础设施）

## 背景

项目工作流 API（`GET /api/v1/projects/{id}/workflow`）已经返回与 LangGraph `StateSnapshot` 模型对齐的快照——`nodes / edges / next / interrupts / values`（源码归属见 `agentteams-controller/internal/server/project_handler.go` 头部注释）。DAG 由团队 Leader 在运行时生成（`plan_dag`），并且可以在项目生命周期内多次重规划。这种动态性是特性，但有一个后果：**转换发生时，没有任何东西记录它。**

当前的具体缺口：

1. **没有转换历史。** 任务状态在对象存储（项目/任务 meta）里只保存当前值。一个任务经历 planned → assigned → in_progress → submitted → revision 之后，只剩 `revision` 存活。排查"工作流为什么卡在这里"目前只能去翻 Matrix 房间历史。
2. **没有节点级检视。** 工作流快照是扁平列表；没有按任务聚合 状态 + spec + 交付物 + 历史 的端点。（`?includeTasks=true` 存在，但返回的是扁平的当前值列表。）
3. **没有图导出。** UI 必须解析 JSON 快照并自行画图。一等公民的 mermaid 导出可以让 dashboard、文档、聊天回复一行渲染整个工作流。
4. **通知不完整。** 任务提交时的代码级完成通知正在推进中（见 #1206）。尚不存在对*所有*转换的统一、可查询的记录。

## 设计方案

四个部分，归入两个 PR。所有状态仍留在对象存储；PR-1 完全不新增写入。

### PR-1 —— 只读可观测性（Controller）

**1. Mermaid 导出。** 扩展工作流端点（例如 `GET /api/v1/projects/{id}/workflow?format=mermaid`），把现有快照渲染成带节点状态类（pending / active / submitted / terminal / interrupted）的 mermaid `flowchart`。纯粹渲染 `buildWorkflow` 已计算好的数据——无新状态、无行为变化。

**2. 节点级状态检视。** 新增按任务的端点（例如 `GET /api/v1/projects/{id}/tasks/{taskId}`），聚合：

- 任务当前状态、指派人、依赖（来自项目 meta）
- 任务 spec/brief（来自任务目录）
- 交付物列表（路径，不含内容）
- append-only 转换 `history[]`（由 PR-2 填充；在此之前为空数组）
- trace 引用：worker 侧 tracing 已经把 entry span 打上 `agentteams.task.id` / `agentteams.project.id` 标签（`plugins/teamharness/adapters/qwenpaw/task_trace.py`），UI 可以从节点深链进 tracing 后端。

### PR-2 —— 引擎纪律（TeamHarness taskflow）

**3. 显式状态转换表。** 任务状态转换的单一事实来源——候选草图（实现前需对照所有转换调用点校验）：

```
planned -> assigned, cancelled
assigned -> in_progress, blocked, cancelled
in_progress -> submitted, blocked, cancelled
submitted -> completed, revision, blocked
completed / revision / blocked / cancelled -> (终态，与 TERMINAL_TASK_STATUSES 一致)
```

在转换收口点强制执行；非法转换以结构化错误拒绝，而不是静默接受。转换表还应写明每个状态的含义（例如 `revision` 在重规划前是终态）。

**4. Append-only 节点历史。** 每个被接受的转换向任务历史追加 `{ts, from, to, actor, action, note?}`，与任务状态一起持久化。大小/保留策略待商定（Q3）。

**5. 转换事件流。** 每个被接受的转换发出结构化事件 `{project_id, task_id, from, to, actor, ts, action, note?}`——持久化、可按游标列表查询（审计 + dashboard），并驱动房间通知（是 #1206 完成通知的超集；见 Q4）。

**6. Worker 进度自报。** 供指派的 worker 在执行中给任务附一条单行进度备注的轻量可选 action（追加进历史、显示在节点面板）。不引起状态变化。

## 边界与非目标

- 对象存储保持为唯一事实来源；PR-1 不新增写入，PR-2 只追加。
- **不做图执行引擎、不做 checkpoint / 时间旅行。** DAG 天生是动态的（LLM 生成、可重规划），replay 语义不适用。历史用于审计与 UX，不用于回滚状态。
- **不与 agent 内部 TODO 模型统一**——层职责不同（会话级工作记忆 vs 项目级共享实体）。
- **不做跨层依赖边同步**；项目 meta 里的 `depends_on` 保持为唯一的跨会话真值。
- 人类干预端点（pause / resume / report）保持现有行为；它们只是成为同一转换表覆盖下的额外转换参与者。

## 与在飞工作的关系

- **#1206（任务完成通知）：** PR-2 的事件流是它的超集。建议：先合 #1206；PR-2 再把通知路由到事件流（或取消该特例）。见 Q4。
- **#1183（持久化任务续跑状态）：** 触碰同样的项目 meta 与 `project_handler.go`。PR-2 的历史持久化必须与其续跑记录共存——两个 PR 落地前应商定存储布局。

## 开放问题

1. **Mermaid 端点形态：** 现有路由加 `?format=mermaid`，还是专用子资源（`/workflow/graph`）？仓库惯例倾向哪种？
2. **事件投递方式：** 持久化游标列表端点是最小集。Controller 是否需要 SSE（或 webhook）支持实时 dashboard，还是 v1 轮询即可？
3. **历史/事件存储与上限：** 追加到每任务状态文件，还是项目级事件日志？仓库里有无保留策略/大小策略先例？
4. **与 #1206 的协调：** 完成通知保持特例，还是成为事件流的第一个消费者？
5. **执行点：** 目前转换由两处写入——TeamHarness MCP 路径（agent 驱动）和 Controller（人类干预）——写的是同一份 meta。转换表应该放在一个共享位置、由两端共同执行，还是 agent 路径收口到 Controller API 做单一执行点？
6. **API 版本化：** 新端点是 additive 挂在 `/api/v1/...` 下，还是版本门控？

## 计划

本 issue 是设计记录。若方向被接受，PR 拆分为：

- **PR-1：** mermaid 导出 + 按任务检视端点（Controller，只读）
- **PR-2：** 转换表 + 节点历史 + 事件流 + 进度自报（TeamHarness，additive）