aws / aws/aws-cdk

(s3tables): support schema evolution for tables defined in CDK

Open
#37,938 4 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-s3tables-alpha effort/medium feature-request p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
1d 19h
Merged PRs (30d)
71

Description

### Describe the feature

Tables defined in CDK don't support schema evolution, there should be a way to add optional fields/expand dtypes for tables that are defined in CDK.

Support for evolution of the partitioning scheme should also be considered.

### Use Case

Schema evolution is an important (really it's table stakes) feature for usage of apache iceberg.

### Proposed Solution

_No response_

### Other Information

_No response_

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### AWS CDK Library version (aws-cdk-lib)

2.250.0

### AWS CDK CLI version

2.1030.0

### Environment details (OS name and version, etc.)

Al2023

Contributor guide

Open the contributing guide

Research direction

Start by tracing the CDK entry point for tables defined in CDK and how it represents Apache Iceberg schemas; the issue names no files or tests. Define supported behavior for adding optional fields and expanding data types, then determine whether partitioning-scheme evolution is in scope and verify the result with the relevant CDK tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.