airbytehq / airbytehq/airbyte

Destination S3: Enhance Relational Database to S3 Connector for Semantic Partitioning

Đang mở
#26,263 3 bình luận 0 reaction 0 người được giao Xem trên GitHub
area/connectors community frozen releaseStage/ga team/destinations type/enhancement
Ngôn ngữ chính
Python
Star
22.1k
Fork
5.3k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

### What area the feature impact?

Connectors

### Revelant Information

**Issue Description:**

Currently, when exporting relational database data to Amazon S3 using the S3 destination connector, there is a limitation in selecting a meaningful column as the "partition key." The connector only supports using a "snapshot date" that does not serve as a suitable partition key for tools such as Hive.

**Expected Behavior:**

It would be beneficial to enhance the S3 Connector to allow users to choose a column as the partition key during data export. This feature would enable users to define a meaningful and semantically relevant partition key that aligns with their specific use cases and analytical requirements.

**Additional Information:**

Semantic partitioning improves query performance by restricting the data scanned to a specific subset relevant to the query.
By leveraging meaningful partition keys, users can optimize their queries and accelerate data processing workflows.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.