lincc-frameworks / lincc-frameworks/hyrax

Use OOP for vector database interactions

Open
#928 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
41
Forks
7
Avg merge
5d 1h
Merged PRs (30d)
8

Description

**Feature request**
The vector database implementations are rather basic at this point. It would be nice to tighten them up. One thing stands out in particular (and a second thing slightly less so):

1. Returning objects that represent database rows. Currently we return quasi-structured dictionaries that are fine for small amounts of work. Returning a list of VectorSearchResult objects (for instance) that have all the attributes of the database would be nice. i.e. an attribute for each column value in the database as well as the vector distance value
2. Turning the query into an object might be nice too. The user can still query by [id || vector] but internally we could convert that into a query object.

If we lean more into Lance as a file format, we should investigate whether lancedb already provides something like this, and if so, perhaps just use that. Of course there are a multitude of other ORM tools that might help as well, but if we go that route we should make extra sure that what we adopt doesn't bring in a slew of new dependecies.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the vector database implementations and reviewing how they currently return dictionary-like results. Investigate whether LanceDB already provides suitable result or query objects, while checking the dependency implications of other ORM options. The scope is complete only after the object model and adoption approach are clearly defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
database, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.