ruvnet / ruvnet/ruflo

[Question] Are agents actually instructed to continuously ingest code into AgentDB and use it for retrieval? Can we track RAG usage?

Open
#1,503 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
72.7k
Forks
8.6k
Avg merge
2d 23h
Merged PRs (30d)
83

Description

## Context

I've been running Ruflo with multi-agent swarms and the AgentDB/HNSW vector memory system. The RAG infrastructure is impressive on paper — ONNX embeddings, HNSW indexing, semantic search, the whole pipeline. But I have a practical concern about **actual adoption by agents during real workflows**.

## The Problem

In my experience, agents almost never:

1. **Ingest code updates into the vector store** after edits, commits, or file changes — unless I explicitly instruct them to do so
2. **Query the vector store for code retrieval** when searching for files, functions, or patterns — they default to `grep`, `find`, or just asking the LLM to guess

The RAG layer feels like it exists in parallel to the actual agent workflow rather than being woven into it. I essentially have to write prompts like:

```
USE MCP TO FEED THIS INTO THE CODE VECTOR STORE
USE MCP TO RETRIEVE FROM VECTOR STORE BEFORE EDITING
```

...which defeats the purpose of having an intelligent orchestration layer.

## Specific Questions

### 1. Are hooks/instructions properly wiring agents to always use AgentDB?

- Do the default agent YAML definitions (coder, reviewer, architect, etc.) include system-level instructions to **always** query the vector store before searching for code?
- Are `post-edit` and `post-command` hooks configured to **automatically** re-ingest modified files into AgentDB embeddings?
- Is there a `pre-edit` hook that forces agents to pull context from the vector store before making changes (similar to what GitNexus does with its PreToolUse hooks)?

### 2. Is there a way to track/audit RAG usage?

I'd love a command like:

```bash
ruflo memory usage --last 24h
# or
ruflo agentdb stats --show-queries
```

That shows:
- How many times agents **stored** embeddings vs. how many files were edited
- How many times agents **queried** the vector store vs. how many search operations occurred
- Hit rate: when agents did query, did they actually use the results?
- Which agents are using RAG and which are bypassing it entirely

### 3. Should there be an "always-on" RAG mode?

Something like:

```yaml
# ruflo.config.json or agent definition
memory:
auto_ingest: true # auto-embed every file touch
force_retrieval: true # agents MUST query vector store before file operations
bypass_penalty: warn # log when an agent skips RAG
```

This would make the RAG layer a **first-class citizen** in the agent loop rather than an opt-in feature that nobody opts into.

## Why This Matters

The whole value proposition of AgentDB + HNSW is that agents build up contextual memory of the codebase over time. But if agents aren't continuously feeding it and aren't forced to consult it, the vector store stays empty (or stale), and the 150x search speedup is irrelevant because nothing is being searched.

For comparison, GitNexus solves this differently — they precompute a full knowledge graph at index time (Tree-sitter AST → call graphs → Leiden clustering → BM25 + semantic hybrid search), and their MCP tools are the **only** way agents access code structure. There's no "opt-in" — the graph IS the interface. Ruflo's approach is more flexible but that flexibility means agents can (and do) just ignore the RAG layer entirely.

## Environment

- Ruflo v3.5.x
- Running via Claude Code (Azure AI Foundry, Opus 4.6)
- Multi-agent swarms (hierarchical topology, 3-8 agents)
- Ubuntu 24.04 remote servers

## Proposed Solutions (any of these would help)

1. **Instrumented hooks** — ship default hooks that auto-ingest on file change and force vector retrieval on search operations
2. **Usage telemetry command** — `ruflo memory stats --verbose` showing ingest/query counts per agent per session
3. **Agent instruction templates** — update default agent YAMLs to include hard instructions about always consulting AgentDB
4. **RAG enforcement mode** — config flag that makes vector store consultation mandatory before file operations, with warnings when bypassed

Would love to hear from the maintainers whether this is by design (agents choose when to use RAG) or a gap in the current hook/instruction wiring. Thanks!

Contributor guide

Open the contributing guide

Research direction

Review the default agent YAML definitions and the post-edit, post-command, and pre-edit hook configuration to determine whether AgentDB ingestion and retrieval are wired into agent workflows. Then inspect the existing memory or AgentDB command entry points and define what usage statistics, per-agent tracking, and enforcement behavior would need to be specified.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.