基线记录(#2985 §14 步骤 1):938d16bf3 上 e2e-full 的 chat-read 46 红/45 绿,其中 43 条是跨 SHA 稳定核;约 36 条首错是「没跑到」不是「不符合契约」
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 0
- Avg merge
- 1h 7m
- Merged PRs (30d)
- 969
Description
> AT 验收方案(#2985)第 14 节第 1 步「先清存量红线:`chat-task-workbench` 与 `e2e-full` 的现有失败先分诊;**记录基线 run,不允许用新增 spec 掩盖**」的产出。本 issue 就是那份**基线记录**。
# 一、绑定的 exact SHA 与基线 run
**`938d16bf3a49c4047f42bbc697833cd921d1053e`** — [run 34155551896](https://github.com/boardx/workspacex/actions/runs/34155551896)(`harness-verify`,push 触发)。
`e2e-full` job 逐步结论:
| step | 结论 |
|---|---|
| Install locked Python runtime | success |
| **Execute uncached trusted full gate** | **failure**(`Failed: @repo/api#test`;Tasks: 20 successful, 21 total) |
| **Execute Chat read/write journey** | **failure**(**46 failed / 45 passed**,30.7m) |
| Execute self-service profile journey | **success** |
| Upload E2E evidence | success |
`chat-task-workbench` job:**skipped**(`workflow_dispatch` 输入 `run_chat_task_workbench` 默认 false,见 `.github/workflows/harness-verify.yml:57-62`)。
# 二、chat-read 车道:46 红 / 45 绿,且这是长期状态
| run | headSha | chat-read |
|---|---|---|
| [34111751233](https://github.com/boardx/workspacex/actions/runs/34111751233) | `a1bd028a1` | 55 failed / 28 passed |
| [34145197536](https://github.com/boardx/workspacex/actions/runs/34145197536) | `0b1d41658` | 44 failed / 47 passed |
| [34152439394](https://github.com/boardx/workspacex/actions/runs/34152439394) | `a8fdf8281` | 48 failed / 43 passed |
| **34155551896(基线)** | **`938d16bf3`** | **46 failed / 45 passed** |
## 稳定核 vs 抖动尾(这是本 issue 最有用的一格)
把基线(`938d16bf3`)与上一轮(`a8fdf8281`)的失败清单按 `spec:line` 取交并集:
- **交集 43 条 = 稳定核**。两个不同 SHA、两次独立运行都红 ⇒ 按方案第 1、11 节判据,这 43 条**不是 flake**,是真实缺口,必须逐条分诊。
- 只在基线红的 2 条:`copilotkit-v2-error-banner.spec.ts:60`、`copilotkit-v2-message-actions.spec.ts:142`。
- 只在上一轮红的 4 条:`agent-task-planning-hitl.spec.ts:186`、`copilotkit-v2-active-file-panel.spec.ts:100`、`copilotkit-v2-attachments.spec.ts:55`、`copilotkit-v2-runtime-adapter.spec.ts:125`。
- 这 6 条是抖动尾,判 flake **需要同 SHA 重跑证据**,本轮没有,先记为「未定性」,不要直接当 flake 忽略。
## 基线红的 46 条按文件分布
| 条数 | spec 文件 |
|---|---|
| 10 | `chat-read.spec.ts` |
| 4 | `copilotkit-v2-hitl.spec.ts` |
| 3 | `copilotkit-v2-tool-rendering.spec.ts` / `chat-attachment-preview-download.spec.ts` / `chat-agent-skill-context.spec.ts` |
| 2 | `copilotkit-v2-runtime-adapter.spec.ts` / `context-engine.spec.ts` / `chat-keyboard-navigation.spec.ts` / `chat-diagram-save-reopen-roundtrip.spec.ts` |
| 1 | `copilotkit-v2-thread-persistence` / `-stream-frame-timing` / `-run-restore-after-switch` / `-roster-landing` / `-persona-archived` / `-message-actions` / `-hitl-dialog-dismiss` / `-error-banner` / `chat-canvas-guidance-render` / `chat-attachment-image-vision-extraction` / `agent-workbench-steering-acceptance` / `agent-workbench-scroll-acceptance` / `agent-workbench-control-acceptance` / `agent-task-planning-hitl` / `agent-task-clarification-result` |
## 首错签名分布(取自 `a8fdf8281` 那轮的 48 条,逐条读过)
| 签名 | 条数 | 说明 |
|---|---|---|
| `expect(locator).toBeVisible/toContainText failed`(元素压根不出现) | ~22 | 主力。含 `chat-read-thread-list` 整个列表拿不到 |
| `locator.fill: Test timeout` / `page.waitForResponse: Test timeout` | ~9 | 输入框都点不到 / 请求永不回 |
| 「4 次尝试都没拿到非空 assistant 气泡」「活动必须在运行流结束前可见」 | ~5 | deep-agent 回环这一轮什么都没产出 |
| 真正走到业务断言并给出期望/实际的 | ~6 | 例:`fill_run_params` 期望 1 条实际 2 条 |
⚠ **按方案第 11 节与本仓纪律,前三类(约 36 条)要先判「没跑到」**:元素不出现、输入框点不到、回环无产出,都不是「行为不符合契约」的证据。**在定位这三类的共因之前,不许把它们逐条记进任何 AT-* 的 `FAIL`**,也不许逐条拆 46 个修复 issue——那会把一个装置问题放大成 46 个假缺陷。
**下一步应该是:先找那约 36 条的单一共因**(web/api/deep-agent 回环起没起来、登录夹具、端口/隔离),再回头看剩下约 6 条真断言。
# 三、`Execute uncached trusted full gate`(`verify:full`)
`Failed: @repo/api#test`,`Test Files 2 failed | 927 passed | 2 skipped (931)`、`Tests 1 failed | 7699 passed | 18 skipped | 2 todo (7720)`。两个失败文件已各自立案:
- #2989 `standard-subtask-tools.test.ts` — `agent_runs.id='parent'` 与 `agent-run/parent-run-control.test.ts` 撞全局主键。**冲突确定性、受害者随机**(上一轮换成 `agui-bridge-sse` 中枪,那个是已知 flake #2968)。
- #2995 `chat-skill-mount-produces-pptx-real-stack.test.ts` T3 反证 — 连红两轮,`expected 1 to be 0`。
# 四、`chat-task-workbench`:基线是「设计上就该红」,**不属于要清的存量红线**
方案第 14 节第 1 步把它和 `e2e-full` 并列,但两者性质不同,记录在此以免后续组按回归门去修它:
- 该 job 在基线 run 里 **skipped**(默认关)。最近一次真跑是 [run 34135593071](https://github.com/boardx/workspacex/actions/runs/34135593071)(headSha `8a3834af7`):**14 failed / 33 passed(47 条,17.8m)**。
- `apps/web/playwright.chat-read.config.ts:210-214` 逐字写明:「这批 spec 的**设计意图就是现在会红**……红的条目 = 尚未实现的差距……不许改成 `test.skip` 来让 CI 变绿。收敛路径是实现能力,不是删断言。」判据单一事实源在 `.harness/instructions/chat-task-workbench-acceptance.md`。
- 那 14 条全是 `TW-*` 记分牌条目(六态工作流指示器、计划面板可编辑、子 Agent 可折叠树、产物四件齐、暂停/恢复/重试/检查点、审批弹窗焦点锁定、点击区 ≥24×24 等)。
⇒ **建议把方案第 14 节第 1 步的措辞改成:「`e2e-full` 的现有失败先分诊;`chat-task-workbench` 只记录基线分数,不作为红线」**,否则后续组会把 14 条能力缺口当成待修的测试 bug。
# 五、相邻已知项(引用,勿重复排查)
#2084(e2e-full 门控空转)、#2968(agui-bridge 共享 30s 预算)、#2983(TC-5 竞态)、#2982(org-platform 全局可变状态)、#2310(chat-read 残余抖动)、#2089 / #2286(端口不靠哈希猜)。
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with .github/workflows/harness-verify.yml:57-62 and the 938d16bf3 baseline run, then read the listed chat-read specs and .harness/instructions/chat-task-workbench-acceptance.md. Investigate the shared cause behind the missing elements, timeouts, and empty assistant output before examining the remaining business assertions. Done means the failures are separated into infrastructure causes, genuine assertions, and unclassified flaky cases without masking them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, playwright, typescript
- Domain
- ci-cd, testing-qa
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100