pingcap / pingcap/tidb

Support inserting and selecting VECTORs as binary data

Open
#59,225 2 comments 0 reactions 0 assignees View on GitHub
type/feature-request
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Feature Request

**Is your feature request related to a problem? Please describe:**

Since VECTOR datatype is implicitly parsed as text as well as converted to text implicitly, there are no way to insert and select VECTOR columns as binary data, causing unnecessary conversion and more network traffic.

**Describe the feature you'd like:**

Comparing to MySQL it is possible creating a program that inserts an array of 32-bit floats instead of STRING_TO_VECTOR() making it more efficient to insert vectors.

Comparing to MySQL and MariaDB the default format when selecting a VECTOR column is binary, so by default it will return the data as 32-bit float array if VECTOR_TO_STRING() or VEC_ToText() is not used. I cannot find any way to get that from TiDB (it also don't support HEX(vector_column)).

**Describe alternatives you've considered:**

**Teachability, Documentation, Adoption, Migration Strategy:**

We should also aim to be compatible with MySQL (and possibly MariaDB) also for VECTOR support.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing TiDB's existing VECTOR behavior around STRING_TO_VECTOR(), VECTOR_TO_STRING(), and VEC_ToText(), then compare binary insert and select behavior with MySQL and MariaDB. Clarify the compatibility expectations, including whether HEX(vector_column) should work, and define completion as efficient float-array insertion and binary VECTOR retrieval without text conversion.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, mysql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.