docToolchain / docToolchain/dacli
RAG via sqlite-vec: Vector search for documentation Q&A
- Dominant language
- Python
- Stars
- 21
- Forks
- 3
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
## Summary
Explore using [sqlite-vec](https://github.com/asg017/sqlite-vec) to enable RAG (Retrieval-Augmented Generation) for documentation Q&A. This builds on top of Issue #185 (SQLite Index) and the lessons from ADR-009 (abandoned iterative LLM ask approach).
## Background
In #186 / ADR-009 we learned that iterating over all documentation files with an LLM is too slow for a built-in `ask` command. RAG offers a better approach: pre-compute vector embeddings for documentation sections, then retrieve only the most relevant chunks for a given question.
## Concept
- **sqlite-vec** (`pip install sqlite-vec`) is a lightweight SQLite extension for vector similarity search (~30MB memory, no external server)
- Combines well with #185: one SQLite database for both structure index and vector embeddings
- Fits our "File System as Truth" principle (ADR-001) — SQLite serves as cache/index only
### Proposed architecture (rough sketch)
1. **Structure Index table** — Section paths, titles, metadata (from #185)
2. **Vector Embeddings table** — One embedding per section/chunk for similarity search
3. **Query flow**: Question → embed → find top-k similar chunks → build context → LLM answers
## Open research questions
- [ ] Which embedding model? Options: Anthropic Embeddings API (hosted), local models like `sentence-transformers` (offline, ~100MB). Trade-off: cost vs. offline capability
- [ ] Chunking strategy: Use existing sections as chunks, or split further? Sections vary widely in size
- [ ] sqlite-vec maturity: Currently v0.1.6 (pre-1.0). Evaluate stability for production use
- [ ] Embedding dimensions: What dimension works best for documentation search? (768, 1024, 1536?)
- [ ] Incremental updates: How to efficiently re-embed only changed sections when docs are updated?
- [ ] Should this be a separate `dacli rag` command, an MCP tool, or integrated into `search`?
## References
- sqlite-vec: https://github.com/asg017/sqlite-vec
- sqlite-vec Python docs: https://alexgarcia.xyz/sqlite-vec/python.html
- ADR-009: Decision against iterative LLM ask feature
- Issue #185: SQLite-based index persistence
- Issue #186: Original ask command (closed, won't implement)
## Related issues
- #185 — SQLite Index (natural foundation for this feature)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.