How to use a Compound Indexes
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 34.7k
- Forks
- 2.7k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 187
Description
We have a collection online,The information is as follows:
curl -X GET 'http://localhost:6333/collections/questions_embeddings_v01_orionstar_text-embedding-bge_39' \
-H 'Content-Type: application/json'
{
"result": {
"status": "green",
"optimizer_status": "ok",
"vectors_count": 461279,
"indexed_vectors_count": 460177,
"points_count": 461279,
"segments_count": 3,
"config": {
"params": {
"vectors": {
"size": 1024,
"distance": "Cosine"
},
"shard_number": 1,
"replication_factor": 1,
"write_consistency_factor": 1,
"on_disk_payload": true
},
"hnsw_config": {
"m": 0,
"ef_construct": 100,
"full_scan_threshold": 10000,
"max_indexing_threads": 0,
"on_disk": false,
"payload_m": 16
},
"optimizer_config": {
"deleted_threshold": 0.2,
"vacuum_min_vector_number": 1000,
"default_segment_number": 0,
"max_segment_size": null,
"memmap_threshold": null,
"indexing_threshold": 20000,
"flush_interval_sec": 5,
"max_optimization_threads": 1
},
"wal_config": {
"wal_capacity_mb": 32,
"wal_segments_ahead": 0
},
"quantization_config": null
},
"payload_schema": {
"metadata.data_id": {
"data_type": "keyword",
"points": 461279
}
}
},
"status": "ok",
"time": 0.000023419
}
He has a payload index "metadata.data_id",But when we use the following query, the query time is very long,About three seconds:
curl --location 'http://10.118.13.232:6333/collections/questions_embeddings_v01_orionstar_text-embedding-bge_39/points/scroll' \
--header 'Content-Type: application/json' \
--data '{
"limit": 4,
"with_payload": true,
"with_vectors": true,
"query_vector": [0.016785502, ......]
"filter": {
"must": [
{
"key": "metadata.source_type",
"match": {
"value": "intervention"
}
},
{
"key": "metadata.data_id",
"match": {
"value": "7bd65f858d2e3144a193b9a3f2db1cdd"
}
},
{
"key": "metadata.is_using",
"match": {
"value": 1
}
}
]
}
}'
So I tried to find a way to add a Compound index, but I couldn't find it,and I tried to add an index for the "metadata.source_type" field,Below is the result of my addition
The same query is executed again, and the discovery takes only a few milliseconds
My problem is to add two separate indexes like the following:
curl -X PUT 'http://localhost:6333/collections/questions_embeddings_v01_orionstar_text-embedding-bge_39'/index \
-H 'Content-Type: application/json' \
--data '{
"field_name": "metadata.data_id",
"field_schema": "keyword"
}'
curl -X PUT 'http://localhost:6333/collections/questions_embeddings_v01_orionstar_text-embedding-bge_39'/index \
-H 'Content-Type: application/json' \
--data '{
"field_name": "metadata.source_type",
"field_schema": "keyword"
}'
Is that equivalent to
alter table add index idx_data_id_source_type(metadata.data_id,metadata.source_type); in mysql?
If not, what makes the same query faster?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the collection information and points/scroll requests in the issue, comparing the payload indexes on metadata.data_id and metadata.source_type with the three-field filter. Clarify whether separate indexes provide compound-index behavior and what accounts for the observed latency difference; document the supported indexing behavior and its limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql
- Domain
- databases, search
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100