docToolchain / docToolchain/dacli

RAG via sqlite-vec: Vector search for documentation Q&A

Open
#265 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
21
Forks
3
Avg merge
1m
Merged PRs (30d)
1

Description

## Summary

Explore using [sqlite-vec](https://github.com/asg017/sqlite-vec) to enable RAG (Retrieval-Augmented Generation) for documentation Q&A. This builds on top of Issue #185 (SQLite Index) and the lessons from ADR-009 (abandoned iterative LLM ask approach).

## Background

In #186 / ADR-009 we learned that iterating over all documentation files with an LLM is too slow for a built-in `ask` command. RAG offers a better approach: pre-compute vector embeddings for documentation sections, then retrieve only the most relevant chunks for a given question.

## Concept

- **sqlite-vec** (`pip install sqlite-vec`) is a lightweight SQLite extension for vector similarity search (~30MB memory, no external server)
- Combines well with #185: one SQLite database for both structure index and vector embeddings
- Fits our "File System as Truth" principle (ADR-001) — SQLite serves as cache/index only

### Proposed architecture (rough sketch)

1. **Structure Index table** — Section paths, titles, metadata (from #185)
2. **Vector Embeddings table** — One embedding per section/chunk for similarity search
3. **Query flow**: Question → embed → find top-k similar chunks → build context → LLM answers

## Open research questions

- [ ] Which embedding model? Options: Anthropic Embeddings API (hosted), local models like `sentence-transformers` (offline, ~100MB). Trade-off: cost vs. offline capability
- [ ] Chunking strategy: Use existing sections as chunks, or split further? Sections vary widely in size
- [ ] sqlite-vec maturity: Currently v0.1.6 (pre-1.0). Evaluate stability for production use
- [ ] Embedding dimensions: What dimension works best for documentation search? (768, 1024, 1536?)
- [ ] Incremental updates: How to efficiently re-embed only changed sections when docs are updated?
- [ ] Should this be a separate `dacli rag` command, an MCP tool, or integrated into `search`?

## References

- sqlite-vec: https://github.com/asg017/sqlite-vec
- sqlite-vec Python docs: https://alexgarcia.xyz/sqlite-vec/python.html
- ADR-009: Decision against iterative LLM ask feature
- Issue #185: SQLite-based index persistence
- Issue #186: Original ask command (closed, won't implement)

## Related issues

- #185 — SQLite Index (natural foundation for this feature)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.