flaky: browser/tests/reasoningbank-adapter.test.ts failed once on an untouched file (run 33973671694 attempt 1); ratchet output loses the assertion on rerun
- Dominant language
- TypeScript
- Stars
- 72.7k
- Forks
- 8.6k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 83
Description
**Observed:** ruflo CI run 33973671694, attempt 1 (PR #3169 at `9dfb751`, `Test Suite (ubuntu-latest)`, job 101326510669): the CI test ratchet flagged one unexpected failing file, `v3/@claude-flow/browser/tests/reasoningbank-adapter.test.ts`. The PR did not touch `v3/@claude-flow/browser`. The session driving the PR re-ran the failed job on the same head; attempt 2 passed (the `test-results-ubuntu-latest` artifact now reads `passed` for that file, report time 15:24:50Z). Second unrelated-file flake of the day after #3207.
**What is lost:** the ratchet prints only the file name, and a rerun replaces the run's `vitest.json` artifact, so the failing test name and assertion from attempt 1 are gone. The job log for attempt 1 has no per-test detail either. That makes this report weaker than #3207's, and it will happen again to the next flake.
**Candidate mechanism (hypothesis, not proven — the assertion was not captured):** `src/infrastructure/reasoningbank-adapter.ts:156` and `:183` build pattern/failure ids as `...-${Date.now().toString(36)}`; two patterns extracted within the same millisecond share an id. The tests at lines 61–91 store trajectories back-to-back and then assert `usageCount > 1` / `totalPatterns > 0`; an id collision in a fast run could flip one of those. `getReasoningBank()` also keeps a module-level singleton (`:273`) while tests construct fresh adapters in `beforeEach` — fine today, but the "zero patterns initially" test at line 168 would break the moment any test switches to the singleton.
**Asks:**
1. Make the ratchet (`scripts/ci-test-ratchet.mjs`) print the failing test names and first failure message per unexpected file, so a rerun cannot erase the evidence.
2. Replace `Date.now().toString(36)` ids with a counter or a proper unique id so same-millisecond patterns cannot collide; or pin the tests' ordering assumptions explicitly.
Until the assertion is captured once, treat this as "flaky, mechanism unconfirmed".
🤖 Generated with [RuFlo](https://github.com/ruvnet/ruflo)
https://claude.ai/code/session_019xHM4rAH4aaShb4DTr1n6s
Contributor guide
Research direction
Start with scripts/ci-test-ratchet.mjs and inspect how unexpected files and vitest.json artifacts are reported. Then read v3/@claude-flow/browser/tests/reasoningbank-adapter.test.ts alongside src/infrastructure/reasoningbank-adapter.ts, especially the cited lines and beforeEach setup. Done means the ratchet preserves failing test names and first messages across reruns, and the confirmed adapter or test-ordering issue no longer flakes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, typescript
- Domain
- ci-cd, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100