onsails / onsails/right-agent

Future work: prefilter recalls Hindsight memory and/or chat history for 'done-before' check

Open
#101 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
35
Forks
4
Avg merge
17h 19m
Merged PRs (30d)
8

Description

Future work: prefilter recalls Hindsight memory and/or chat history for "done-before" check

Context

Today the skill-learning prefilter (crates/bot/src/learning_prefilter.rs::build_prompt) decides skip / patch_existing / create_new from a snapshot of the just-finished turn:

  • user_msg_text (≤2000 chars)
  • assistant_reply_text (≤4000 chars)
  • used_skill_receipts
  • existing rightx-* skill index (name + ≤120-char description)
  • per-turn cost / num_turns / elapsed vs agent's P50/P90/P99 baseline

That's enough to spot trivial chats and existing-skill reuse, but it has a blind spot: the agent may have solved the same problem days ago without producing a skill receipt, and the prefilter has no signal for that. Result: a create_new decision for a procedure that's already in Hindsight memory or recent chat history, and a redundant probe-writer run downstream.

Proposal

Add a lightweight "have we done this before?" lookup to the prefilter input:

  1. Hindsight recall. Issue a short memory_recall query keyed on the topic of the turn (extracted from the user msg or assistant reply) and pass the top N (≤3) hits into the prompt as PRIOR PROCEDURE EVIDENCE.
  2. Chat history search. Optionally run thread_search / chat_search against the current chat for similar prior turns. Pass back compact snippets (caller, ts, ≤200 chars) so Haiku can see "we already walked the user through this on date X".
  3. Decision impact. Reword the framing so that prior evidence biases toward patch_existing (if a matching skill exists) or skip (if the procedure was handled inline and is too narrow to deserve a skill yet). Keep create_new for genuinely novel work.
Considerations
  • Latency / cost. Recall + search adds round trips and tokens. Cap to single-shot queries with hard byte budgets so the prefilter stays inside its 30s timeout and the Haiku prompt stays small.
  • Daily learning budget. Whatever recall/search costs should be counted against LEARNING_SOURCES so today_spend_usd still gates correctly.
  • False suppression. A naive recall match can suppress legitimately new work that just happens to share keywords. Make the prompt explicit: prior evidence is a hint, not a verdict — Haiku still decides.
  • Schema. No new decision values needed. Optionally extend reason so we can audit which prior-evidence link triggered a patch_existing or skip.
Out of scope
  • Changing the curator / probe-writer side. This is strictly a prefilter input enrichment.
  • Cross-agent recall. Stay scoped to the current agent's own memory and chat for now.
Files
  • crates/bot/src/learning_prefilter.rsbuild_prompt, schema extension if needed
  • crates/bot/src/learning_pipeline.rs — wiring of the new input
  • crates/right-agent/src/usage.rs — accounting for the recall/search cost

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in crates/bot/src/learning_prefilter.rs at build_prompt, then trace input wiring through crates/bot/src/learning_pipeline.rs and cost accounting in crates/right-agent/src/usage.rs. Determine how bounded Hindsight and chat-history evidence can reach the prefilter while preserving the existing decisions and timeout; done means prior evidence is considered, costs are gated, and false suppression is addressed without changing curator or probe-writer behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.