[CH] support customized file scan which does not support transform
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
When we use customized file scan( not support transform ) , the output spark columnar format data cannot be passed to a transform supported exec directly.
**Describe the solution you'd like**
A columnar(spark) to columnar(backend native) plan ts needed between file scan and its transform supported parent
**Describe alternatives you've considered**
Otherwise we can use a step combined of C2R and R2C instead
For C2R and R2C are already implemented
Contributor guide
Research direction
Start by reviewing the existing C2R and R2C implementations mentioned in the issue, then trace how a customized file scan connects to a transform-supported parent. The requested result is a columnar Spark-to-native plan between those operators without requiring a transform-unsupported scan.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, scala, spark
- Domain
- backend, data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100