cube-js / cube-js/cube

[Feature request] Add support to Read Athena Results from S3 Using DuckDB

Open
#9,267 2 comments 0 reactions 0 assignees View on GitHub
driver:athena
Dominant language
Rust
Stars
20.8k
Forks
2.1k
Avg merge
1d 2h
Merged PRs (30d)
181

Description

Currently, Athena Drive relies solely on the AWS API, which is great for access control management. However, when handling large query results in streaming mode, performance can be significantly impacted due to the REST API's limitation of returning a maximum of 1,000 rows per request.

It would be beneficial to have an optional mode to read the generated CSV result directly from S3 using DuckDB, which is extremely fast and would allow fetching the entire result with a single request instead of iterating over multiple paginated responses.

This would not replace the default behavior but serve as an opt-in alternative for scenarios where performance is a concern.

If this makes sense, I'm happy to contribute with a PR for this feature. Let me know your thoughts!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.