OWASP / OWASP/OpenCRE

Chat embeddings: retrieve heading/harvest chunks, not whole-node blobs

Open
#1,076 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement not ready for development untriaged
Dominant language
Python
Stars
180
Forks
137
Avg merge
3d 23h
Merged PRs (30d)
21

Description

Task

Chat and import embeddings are still one vector per node, often the whole page. That is why AI Exchange (and similar long markdown) comes back as a single blob. After #1075 returns top-k hits, those hits are still coarse unless we store section-level embeddings_content.

Harvester already has markdown_heading / Docling strategies (application/utils/harvester/chunker.py). Chat does not read harvest_input / chunk records; it reads the embeddings table.

Success criteria (all must pass)

  • Document the current path: PromptHandler.generate_embeddingsembeddings.embeddings_content (whole node / fetched URL) vs Module A chunks
  • Design (in-issue or short RFC comment): either (a) embed heading-sized slices into embeddings with a stable embeddings_url fragment, or (b) chat retrieval may read harvested chunk text when a node has OIE chunks — pick one, do not run two indexes
  • AI Exchange (and other long MD standards) no longer injects an entire page as the only context for a narrow question
  • embeddingsUrl / citations still resolve to a section, not just the document root
  • Heroku: no torch / LlamaIndex on prod slug. Chunking for import may stay on workers / requirements-dev.txt
  • Tests for the chosen slice boundary (heading vs fixed-size) and for citation URL fragments
  • make lint / make mypy / targeted tests green

Context

  • Depends on / follows #1075 (multi-hit chat is wasted if each hit is the whole AIX page)
  • Harvester: application/utils/harvester/chunker.py, docling_chunker.py, repos.yaml
  • Embed path: application/prompt_client/prompt_client.py (generate_embeddings, normalize_embeddings_content)
  • Related: #962, OIE Module A

Constraints

  • In scope: finer-grained chat/import embeddings for long standards
  • Out of scope: replacing chat with a notebook product; PDF-only RAGFlow; changing Module C’s CRE hub identity
  • Do not duplicate Module A’s LlamaIndex Docling onto the web dyno

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with application/prompt_client/prompt_client.py, especially generate_embeddings and normalize_embeddings_content, then compare the existing strategies in application/utils/harvester/chunker.py and docling_chunker.py. Choose one indexing approach and document the design before implementing it. Done means section-level retrieval and citation fragments work for long Markdown standards, with boundary and URL-fragment tests plus make lint, make mypy, and targeted tests passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.