Support DuckDB as computation backend
- Dominant language
- Kotlin
- Stars
- 1.1k
- Forks
- 83
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 30
Description
If I understand Kotlin dataframes correctly, computations are done directly in the JVM.
For many use-cases, the optimized data storage and vectorized computations of DuckDB could be very useful in terms of performance and memory consumption.
So maybe there could be a way to move computations to DuckDB via Apache Arrow, which offers a columnar memory format, zero-copy reads, streaming, and an extensive Java API.
The goal would be to perform analytical computations against DuckDB directly from Kotlin in a way similar to the one described here for Python and R:
https://arrow.apache.org/blog/2021/12/03/arrow-duckdb/
Looking forward to hear what you think about the feasibility of such an approach.
Contributor guide
Assessment
This issue has not been assessed yet.