Implement Spark bootstrap for IVF clustering and MDT posting generation
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Parent: #19094
Related: #18676
## Summary
Implement Spark bootstrap for IVF clustering and MDT posting generation.
## Scope
- centroid training
- vector assignment to centroids
- posting generation
- quantizer and manifest rows
- generation write path
## Acceptance Criteria
- index bootstrap produces a complete generation
- bootstrap output is persisted to MDT
- bootstrap logic is separate from read-path logic
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading parent issue #19094 and related issue #18676, since the payload names no files, tests, or code entry points. Trace the requested centroid training, assignment, posting, quantizer, manifest, and generation write-path components, and consider the work complete when a full index generation is produced and persisted to MDT with bootstrap logic separate from read-path logic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering, distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100