biocompute-objects / biocompute-objects/bco-rag
Semantic Chunking Chunk Size Bug
Open
bug
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Llamaindex's `SemanticSplitterNodeParser` can sometimes produce chunks that are too large for the embedding model. Unfortunately there is no max length option for the semantic chunking to avoid this issue.
Will have to eventually subclass the `SemanticSplitterNodeParser` and create a two level safety net that will naively split large chunks into sub-chunks in order to stay under the embedding model input token limits.
Reference:
https://github.com/run-llama/llama_index/issues/12270
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.