Support vector index on multiple columns
- Dominant language
- Go
- Stars
- 24.4k
- Forks
- 873
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 108
Description
Currently our vector index workflow is based on MariaDB, which defines vector indexes on a single column that contains the entire vector. We currently support vector indexes on JSON columns and plan to add support for a dedicated `VECTOR` type.
However, some vector datasets on HuggingFace take a different approach and store each dimension of the vector in a different float column. Importing these datasets into Dolt in a way that makes them indexible requires extra steps to convert the representation of the data.
But there's no reason why we shouldn't be able to support vector indexes on multiple float columns. For example:
```
CREATE TABLE test(pk int primary key, v0 float, v1 float, v2 float, v3 float);
CREATE INDEX vidx on TEST(v0, v1, v2, v3);
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.