[Feature][transform-v2] Vector Functions Support in SQL Transform
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 210
Description
### Search before asking
- [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
### Description
Add comprehensive vector processing capabilities to SeaTunnel's SQL Transform engine, enabling users to perform vector operations directly within SQL queries for machine learning and AI workloads.
### Core Vector Functions
1. **VECTOR_REDUCE** - Vector dimensionality reduction with multiple algorithms
2. **VECTOR_NORMALIZE** - Normalize vector to unit length
### Supported Reduction Methods
- **TRUNCATE** - Simple truncation: keep first N dimensions
- **RANDOM_PROJECTION** - Gaussian random projection
- **SPARSE_RANDOM_PROJECTION** - Sparse random projection for memory efficiency
### Usage Scenario
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating SeaTunnel's SQL Transform engine and its existing function registration and vector-related handling. Review how transform functions are defined and tested, then define support for VECTOR_REDUCE with the listed reduction methods and VECTOR_NORMALIZE; done means these operations work directly in SQL queries with coverage for each method.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- ai, data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100