AnswerDotAI / AnswerDotAI/RAGatouille

Turn Off return of the verbose unstructured text from --> result = RAG.search(query=query,k=4)

Open
#241 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4k
Forks
276
PR merge metrics
No merged PRs in 30d

Description

How to turn off return of -- the verbose unstructured text -->
using --> result = RAG.search(query=query,k=4)
and just Return the dict -> [{"content": ...

[Aug 18, 16:57:05] Loading segmented_maxsim_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...
Loading searcher for index ragcorpus for the first time... This may take a few seconds
[Aug 18, 16:57:06] #> Loading codec...
[Aug 18, 16:57:06] #> Loading IVF...
[Aug 18, 16:57:06] Loading segmented_lookup_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...
[Aug 18, 16:57:06] #> Loading doclens...
[Aug 18, 16:57:06] #> Loading codes and residuals...
[Aug 18, 16:57:06] Loading filter_pids_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...
[Aug 18, 16:57:06] Loading decompress_residuals_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...
Searcher loaded!

#> QueryTokenizer.tensorize(batch_text[0], batch_background[0], bsize) ==
#> Input: . who is Forneus?, True, None
#> Output IDs: torch.Size([32]), tensor([ 101, 1, 2040, 2003, 2005, 2638, 2271, 1029, 102, 103, 103, 103,
103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103,
103, 103, 103, 103, 103, 103, 103, 103])
#> Output Mask: torch.Size([32]), tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0])

***
I just want this -->
[{"content": ...

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the RAG.search(query=query, k=4) entry point and reproduce the verbose loading and tokenizer output shown in the issue. Determine how output is produced and whether it can be disabled while preserving the requested [{"content": ...}] result; done means the search returns that result without the unwanted verbose text.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.