apache / apache/hudi

Add vector query planning using centroids and cluster probing

Open
#19,101 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Parent: #19094
Related: #18676

## Summary
Add vector query planning using centroids and cluster probing.

## Scope
- load manifest, centroid, and quantizer metadata
- select probe clusters
- derive posting ranges to scan
- support query planning inputs needed by the Spark path

## Acceptance Criteria
- planning logic is isolated from scoring and reranking
- probe selection behavior is explicit
- posting ranges are derived from metadata state

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with parent issue #19094 and related issue #18676, then trace the Spark path and its query-planning entry points. Review how manifest, centroid, quantizer, and posting-range metadata are represented. Done means isolated planning logic with explicit probe selection and ranges derived from metadata, without scoring or reranking.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
data-engineering, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.