CatchTheTornado / CatchTheTornado/text-extract-api

[feat] `vector_strategies` for vector db indexing

Open
#100 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
3.2k
Forks
279
PR merge metrics
No merged PRs in 30d

Description

It would be really cool to add indexing strategies for this data later on. Right now we have `storage_strategies`, which is fine, but I'm thinking of something like `vector_strategies` with `pineconedb`, `pg_vector`, and similar approaches—so that PDF documents are stored directly in a vector database. How awesome would that be? Then you could immediately set up a RAG (Retrieval-Augmented Generation) workflow!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the existing `storage_strategies` implementation and its tests or entry points. From there, determine how indexing strategies are configured and what integration points would be needed for the proposed vector-database approaches. Done would require an agreed scope and implementation plan for supported vector stores and RAG-oriented document storage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.