apache / apache/datafusion-comet
`native_datafusion` performance improvement
Open
enhancement
native_datafusion
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 375
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
### What is the problem the feature request solves?
Comparing to `native_iceberg_compat` the `native_datafusion` takes twice more memory on pods and after #3046 the scan grew slower, prev it was at least 20% faster. It might be a problem with aggressive SchemaAdapter transformations
### Describe the potential solution
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.