ruvnet / ruvnet/ruflo

[Dream Cycle 2026-09-02] intelligence: LearningBridge.consolidate() reward-blind fix + capabilities,memory scan

Open
#3,159 0 comments 0 reactions 0 assignees View on GitHub
capabilities dream-cycle intelligence memory research
Dominant language
TypeScript
Stars
72.7k
Forks
8.6k
Avg merge
2d 23h
Merged PRs (30d)
83

Description

## 1. Tonight's Rotation

```
DATE=2026-09-02
DEEP=intelligence
SCAN=capabilities,memory
SLOT=2 (DAYINT % 5)
COMMIT=4d0134e59b4fa5e8552cb7b98c6b9846f08b0c82
```
No bonus deep dive (DAYINT%25=2, %75=27 — neither trigger).

## 2. Ledger Check

Last 14 rows inspected. Prior gist (2026-09-01, security/turnWindowMs) scored **10/10** on the STEP 1.2 rubric: 8 competitor rows, a primary-source MCP-spec read, valid witness, executable recommendations, under 1500 words.

**Two confirmed, distinct findings from the ledger check itself:**
- **2026-08-24 → 2026-09-01 (9 nights): ledger-append bug.** All 9 nights ran the full pipeline (branch, issue, evaluated draft PR each — verified via `git ls-remote` and a GitHub PR listing, not inferred) but `docs/dream-cycle/LEDGER.md` was never appended. Same failure mode as the 08-14..08-19 gap recovered on 08-19. Backfilled tonight; root cause of the append step itself remains undiagnosed — flagged as a standing candidate for a future `automation`/`meta` scan.
- **2026-08-20 → 2026-08-23 (4 nights): real pipeline gap**, not a ledger bug — no `dream/*` branches, issues, or PRs exist for these dates at all.

No intelligence-surface finding repeated 3+ times in the trailing window, so no reject-duplicate trigger.

## 3. Deep Dive Findings (intelligence)

5 parallel research roles (Deep Researcher, Scan A/capabilities, Scan B/memory, Competitor Analyst, Ruflo Architecture Reviewer) converged on: `LearningBridge.consolidate()` (`v3/@claude-flow/memory/src/learning-bridge.ts`) hardcoded `completeTask(trajectoryId, 1.0)` for every completed trajectory, regardless of the underlying insight's actual quality — a reward-blind learning path. Full write-up: `docs/dream-cycle/dream-gist-2026-09-02.md`.

**Separately** (not tonight's candidate, a governance finding): PR #3110 (2026-08-27, EWC-gate wiring fix, already evaluated ACCEPT-scoped) was never merged to `main` — that bug is still live in production. Recommend prioritizing its merge.

## 4. Hypothesis

> Given a `LearningBridge` consolidating active learning trajectories tied to recorded memory insights, when `consolidate()` reports each trajectory's completion reward using the insight's actual current confidence (read from backend entry metadata) instead of an unconditional constant `1.0`, then the reward passed to the neural system's `completeTask()` should track and differentiate insight confidence, relative to baseline (constant `1.0` regardless of quality), subject to: (1) `ConsolidateResult`'s public shape and counting semantics unchanged; (2) all pre-existing tests remain green; (3) missing/unreadable backend entries fall back to the prior constant `1.0`; (4) fully deterministic, $0 test coverage.

## 5. Evaluation Receipt

Real evaluator: `vitest run` on `v3/@claude-flow/memory/src/learning-bridge.test.ts`, deterministic, $0.

- Baseline (pre-diff, stashed): 56/56 passing.
- Candidate (final, post-critique-fix): **62/62 passing** (56 unchanged + 6 new).
- Full package regression: 464/465; the 1 failure is a pre-existing, environmental read-only-file-permission test (root bypasses chmod in this sandbox) — reproduced identically on baseline.

## 6. Darwin Results

Not run, deliberately — this is a binary correctness fix with no tunable parameter, same reasoning as PR #3152's precedent.

## 7. Flywheel Evidence

No `.claude-flow/flywheel/` state in this repo. Evidence retained as: the diff, this issue, the PR, and `docs/dream-cycle/dream-gist-2026-09-02.md`.

## 8. Reward Hack Check

Manual checklist (no generic diff/benchmark-scanning CLI reachable this session). No test weakened — diff only adds coverage (confirmed by the clean 56→62 baseline/candidate comparison). No gold data touched. No seed/metric manipulation. Minor added cost: one `backend.getByKey()` call per active trajectory in `consolidate()`, gated by `consolidationThreshold` (default 10) — not hidden, noted here.

## 9. Security Review

Not security-sensitive: internal learning-signal correctness fix, no new attack surface, no credential/network/filesystem exposure change.

**Independent adversarial-critic subagent (separate context) — two rounds:**
- **Round 1: found a real bug in the fix itself.** `entryId` flowing through `LearningBridge` is the caller-assigned `key` string (from `AutoMemoryBridge.storeInsightInAgentDB`), not the backend's internal `entry.id` (an independently-generated UUID). The original fix called `backend.get(entryId)`, which is indexed by `id`, so it would almost always miss in production and silently fall back to the same `1.0` the fix claimed to eliminate — passing unit tests only because the mock backend's `get()` didn't validate id/key correspondence.
- **Fixed:** switched to `backend.getByKey(namespace, key)` (traced and confirmed correct across all three real backends — AgentDB, SQLite, hybrid), added a configurable `insightNamespace` (default `'learnings'`), and rewrote the mock backend's `getByKey` to do a genuine namespace+key lookup so the tests exercise the real id≠key distinction via `backend.store()` + `onInsightRecorded(insight, entry.key)`, not a permissive stub.
- **Round 2 verdict: CONFIRMED**, independently re-traced against `agentdb-backend.ts`, `sqlite-backend.ts`, and `hybrid-backend.ts`. One non-blocking nit: `controller-registry.ts` instantiates a second `LearningBridge` that doesn't thread `insightNamespace` through, but that instance is never fed any insights (dead/unwired in that path) — pre-existing, out of scope tonight.

## 10. Scan Findings: capabilities

Ruflo's plugin system has a fully-typed `PluginPermission` capability manifest (`network`/`filesystem`/`execute`/etc.) plus install-policy shape (`allowedPermissions`, `minTrustLevel`) — but grepping the actual loader (`plugins/manager.ts`, `plugins/store/index.ts`) finds **zero consumers** of any of it; `manager.ts:341` explicitly logs "Plugins run with full process access." The trust-anchor Ed25519 key is still a literal `PLACEHOLDER`. This is ahead of bare MCP (which has no native per-tool authorization at all per current OWASP/MCP-gateway guidance) in *shape*, but the enforcement layer is missing — worth a HIGH-severity flag in a future `metaharness mcp-scan`, distinct from the already-tracked HIGH-04 unsandboxed-execution finding.

## 11. Scan Findings: memory

Ruflo's memory layer is more advanced than expected going in: RRF fusion, recency-decay, and MMR diversity re-ranking already exist in `smart-retrieval.ts`. The concrete gap: filtering (`hnsw-index.ts:388`, `agentdb-backend.ts`) is post-filter/over-fetch, not graph-aware — ACORN (SIGMOD 2024, Grade A) and its 2025-26 follow-ons (RACORN-1, Compass) show graph-native filtered HNSW can be 2-1000x faster at fixed recall under selective filters. Weaviate shipped ACORN-style filtering (v1.27+, default-on v1.34); Qdrant ships a competing filterable-HNSW approach. Not proposed as tonight's candidate — flagged for a future night.

## 12. Competitors Reviewed

LangGraph, AutoGen/MS Agent Framework, CrewAI, OpenAI Agents SDK/AgentKit, Qdrant, Weaviate, Milvus, LanceDB, Vespa, MCP (protocol). Full table + "why the evolutionary-loop column is uniformly empty" analysis in the gist — headline: CrewAI explicitly declined this design (issue #3015, closed not-planned) and OpenAI is actively retreating from it (Agent Builder/Evals wind-down announced 2026-06-03), both citing/evidencing safety concerns (DGM reward-hacking, documented "capability erosion under self-evolution") that Ruflo's Darwin/Flywheel governance shape is specifically built to address.

## 13. Gist

`docs/dream-cycle/dream-gist-2026-09-02.md` (committed to the repo — no gist-creation tool was reachable in this execution environment, only GitHub issue/PR/repo MCP tools; noting this honestly rather than fabricating a gist URL).

## 14. Witness

| Field | Value |
|---|---|
| Session commit | `4d0134e59b4fa5e8552cb7b98c6b9846f08b0c82` |
| Report SHA-256 (pre-witness content) | `29c9189c833c3d56c378f9a1329b3baf6f16543493ad28e4e62ec76b14c0fdb9` |
| Witness stamp | `bead6402cb018f0e57ec6695c82e007365b5f2ae91d530a74c82a2eb8a1d362b` |

## 15. Recommendation

evaluated: accepted

1. Merge this fix.
2. **Merge PR #3110** — an already-evaluated, still-open EWC-gate fix with a live production defect.
3. **Diagnose the ledger-append failure mode** (9 recurrences now) as a future automation/meta finding.

Contributor guide

Open the contributing guide

Research direction

Start with v3/@claude-flow/memory/src/learning-bridge.ts and its learning-bridge.test.ts, then trace getByKey through agentdb-backend.ts, sqlite-backend.ts, and hybrid-backend.ts. Run the listed Vitest file first. Done means rewards reflect stored insight confidence, missing or unreadable entries retain the 1.0 fallback, ConsolidateResult semantics stay unchanged, and the deterministic tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
databases, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.