aws-samples / aws-samples/startup-data-pipeline-samples
Update pipline to stream the CDC data into iceberg tables using Firehose
Open
enhancement
- Dominant language
- TypeScript
- Stars
- 16
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Update new pipeline to capture CDC from RDS and stream the data into Iceberg tables on Amazon S3 using Amazon Firehose.
Use following feature:
https://aws.amazon.com/jp/blogs/aws/replicate-changes-from-databases-to-apache-iceberg-tables-using-amazon-data-firehose/
---
現状Amazon AuroraからS3 ExportでS3に出力しているフローとは別に、先日リリースされたFirehose CDC streaming into iceberg tablesを利用してリアルタイムでのデータ更新に対応する
Contributor guide
Assessment
This issue has not been assessed yet.