AnswerDotAI / AnswerDotAI/RAGatouille

llama_index_sentence_splitter issues

Open
#66 2 comments 0 reactions 0 assignees View on GitHub
unclear/needs more details
Dominant language
Python
Stars
4k
Forks
276
PR merge metrics
No merged PRs in 30d

Description

ragatouille 0.0.4b2 , ubuntu 22.04

when I using the sample code to run , documents is just a list of string .

Traceback (most recent call last):
File "/workspace/three_methods_ranking2.py", line 160, in
my_documents = processor.process_corpus(documents)
File "/usr/local/lib/python3.10/dist-packages/ragatouille/data/corpus_processor.py", line 22, in process_corpus
documents = self.document_splitter_fn(documents, **splitter_kwargs)
File "/usr/local/lib/python3.10/dist-packages/ragatouille/data/preprocessors.py", line 9, in llama_index_sentence_splitter
docs = [[Document(text=doc)] for doc in documents]
TypeError: 'NoneType' object is not iterable

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with ragatouille/data/preprocessors.py, especially llama_index_sentence_splitter, then trace its call from data/corpus_processor.py through process_corpus. Reproduce the sample using a list of strings on Ubuntu 22.04 and the reported version; done means the corpus is processed without the NoneType iteration error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.