aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap
Feature request for adding CFN support to UpdataTables API for Glue Iceberg tables
- Dominant language
- No language data
- Stars
- 1.1k
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
### Name of the resource
AWS::Glue::Table
### Resource name
_No response_
### Description
### Describe the feature
Hi team, I want to ask for a feature request based on the following behaviour faced by a customer:
> We are using the Glue CfnTable L1 construct in CDK to define Iceberg tables. When we try to update the schema of these tables in our CDK code and deploy the changes, all the iceberg-specific glue table properties (such as table_type = "ICEBERG" and metadata_location = "S3://" ) are removed from the table, resulting in the inability to query or write to the table via Athena or the firehose that feeds the table.
> originally reached out to AWS support about this and was told that the UpdateTable API was not yet supported for Iceberg tables in the support case 174888831500226
On top of this behaviour, the link [1] mentions this note: **The UpdateTable API does not currently support Iceberg table format as an input to the operation.**
However, I have confirmed that the [Glue UpdateTable API](https://docs.aws.amazon.com/glue/latest/webapi/API_UpdateTable.html) is in fact compatible with Iceberg tablea. This can be in fact quickly verified by editing an Iceberg table in the AWS console and checking the corresponding UpdateTable event in CloudTrail completing successfully. However, CDK and CloudFormation have not been updated to expose this capability natively.
I want to ask for the Glue [CfnTable L1 construct in CDK](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_glue.CfnTable.html) and the related [CloudFormation components](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-glue-table.html) to allow the UpdateTable behaviour as freely as the Glue UpdateTable API and also updated the doc [1] since the note is incorrect.
Attached is a workaround [2] using [AwsCustomResource in CDK](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.custom_resources.AwsCustomResource.html), which gives full control over the Glue API and allows us to create and update Iceberg tables accordingly without any of the metadata loss issues previously encountered.
[1] - https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-iceberg.html
[2] - [glue-table-stack.js](https://github.com/user-attachments/files/26605025/glue-table-stack.js)
### AWS CDK Library version (aws-cdk-lib)
aws-cdk-lib@2.246.0
### AWS CDK CLI version
2.1115.1 (build e979d8d)
### Environment details (OS name and version, etc.)
macOS Tahoe 26.4
### Other Details
Previously approached CDK team to request this but they recommended to now approach CloudFormation team via this medium: [+] - https://github.com/aws/aws-cdk/issues/37560#issuecomment-4230667999
Contributor guide
Research direction
Start with the AWS::Glue::Table CloudFormation resource documentation and the Glue UpdateTable API behavior described in the issue, then compare them with the CDK CfnTable L1 construct and the attached glue-table-stack.js workaround. Done means Iceberg-specific properties survive schema updates through CloudFormation/CDK and the incorrect Iceberg UpdateTable note is corrected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100