Azure-Samples / Azure-Samples/chat-with-your-data-solution-accelerator

Implement more efficient chunking strategies

Open
#2,103 2 comments 1 reaction 0 assignees View on GitHub
enhancement stale
Dominant language
Python
Stars
1.2k
Forks
648
Avg merge
1d 30m
Merged PRs (30d)
9

Description

# Motivation

Currently the chunking is possible using Layout, Page, Fixed-size and Paragraph strategies with possible overlap. I would suggest an additional strategy focused solely on quality: LLM chunking. A LLM is called to have coherent and relevant chunks, each chunk expressing the same idea or concept or thought.

In all chunking strategies, a LLM can be used to generate additional metadata to improve the reranking by Azure AI Search when semantic search is True. The LLM would populate for each chunk:
- "Title" by summarizing the chunk into one short sentence
- "Keyword" by extracting the main keywords of the chunk

Please note that in order to make the reranking useful, [ticket 2093](https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/issues/2093) needs to be implemented.

# Tasks

_To be filled in by the engineer picking up the issue_

- [ ] Task 1
- [ ] Task 2
- [ ] ...

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing Layout, Page, Fixed-size, and Paragraph chunking strategies and the Azure AI Search semantic-search path. Define the LLM chunking and per-chunk Title/Keyword metadata requirements, then check how ticket 2093 affects reranking. Done should include concrete tasks and an agreed implementation scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
ai, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.