tortoise / tortoise/tortoise-orm
Add vector field (pgvector, sqlite-vss, etc)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 516
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 9
Description
TL;DR make it possible to use tortoise-orm as a vector database
Is your feature request related to a problem? Please describe.
A lot of commercial vector databases are being thrown around but none of them adequately support your own schemas and never have the ability to use an ORM, pgsql+pgvector is probably the best choice so it would be good to support it in tortoise-orm .
Describe the solution you'd like
Add a new vector field, if using a postgresql database, it uses pgvector and if using sqlite it uses sqlite-vss.
Ideally it would allow adding/updating an index after rows have been added (recreate possible?)
Describe alternatives you've considered
The only real alternative is to not use tortoise-orm
Additional context
Vectors are used to store embeddings, such as those coming from OpenAI, they can be used to retrieve documents based on their semantic similarity, images by content and so on.
Links:
https://towardsdatascience.com/explaining-vector-databases-in-3-levels-of-difficulty-fc392e48ab78
https://platform.openai.com/docs/guides/embeddings
https://github.com/pgvector/pgvector
https://github.com/asg017/sqlite-vss
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing tortoise-orm's field and PostgreSQL/SQLite backend extension points, then compare the pgvector and sqlite-vss requirements named in the issue. Define the vector field's storage and index behavior for both databases, including how indexes are added or recreated. Done means support is tested for both backends.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python, sqlite
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100