Chat embeddings: retrieve heading/harvest chunks, not whole-node blobs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 180
- Forks
- 137
- Avg merge
- 3d 23h
- Merged PRs (30d)
- 21
Description
Task
Chat and import embeddings are still one vector per node, often the whole page. That is why AI Exchange (and similar long markdown) comes back as a single blob. After #1075 returns top-k hits, those hits are still coarse unless we store section-level embeddings_content.
Harvester already has markdown_heading / Docling strategies (application/utils/harvester/chunker.py). Chat does not read harvest_input / chunk records; it reads the embeddings table.
Success criteria (all must pass)
- Document the current path:
PromptHandler.generate_embeddings→embeddings.embeddings_content(whole node / fetched URL) vs Module A chunks - Design (in-issue or short RFC comment): either (a) embed heading-sized slices into
embeddingswith a stableembeddings_urlfragment, or (b) chat retrieval may read harvested chunk text when a node has OIE chunks — pick one, do not run two indexes - AI Exchange (and other long MD standards) no longer injects an entire page as the only context for a narrow question
-
embeddingsUrl/ citations still resolve to a section, not just the document root - Heroku: no torch / LlamaIndex on prod slug. Chunking for import may stay on workers /
requirements-dev.txt - Tests for the chosen slice boundary (heading vs fixed-size) and for citation URL fragments
-
make lint/make mypy/ targeted tests green
Context
- Depends on / follows #1075 (multi-hit chat is wasted if each hit is the whole AIX page)
- Harvester:
application/utils/harvester/chunker.py,docling_chunker.py,repos.yaml - Embed path:
application/prompt_client/prompt_client.py(generate_embeddings,normalize_embeddings_content) - Related: #962, OIE Module A
Constraints
- In scope: finer-grained chat/import embeddings for long standards
- Out of scope: replacing chat with a notebook product; PDF-only RAGFlow; changing Module C’s CRE hub identity
- Do not duplicate Module A’s LlamaIndex Docling onto the web dyno
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with application/prompt_client/prompt_client.py, especially generate_embeddings and normalize_embeddings_content, then compare the existing strategies in application/utils/harvester/chunker.py and docling_chunker.py. Choose one indexing approach and document the design before implementing it. Done means section-level retrieval and citation fragments work for long Markdown standards, with boundary and URL-fragment tests plus make lint, make mypy, and targeted tests passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100