aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap

Feature request for adding CFN support to UpdataTables API for Glue Iceberg tables

Open
#2,495 0 comments 1 reaction 0 assignees View on GitHub
Coverage
Dominant language
No language data
Stars
1.1k
Forks
62
PR merge metrics
No merged PRs in 30d

Description

### Name of the resource

AWS::Glue::Table

### Resource name

_No response_

### Description

### Describe the feature

Hi team, I want to ask for a feature request based on the following behaviour faced by a customer:
> We are using the Glue CfnTable L1 construct in CDK to define Iceberg tables. When we try to update the schema of these tables in our CDK code and deploy the changes, all the iceberg-specific glue table properties (such as table_type = "ICEBERG" and metadata_location = "S3://" ) are removed from the table, resulting in the inability to query or write to the table via Athena or the firehose that feeds the table.

> originally reached out to AWS support about this and was told that the UpdateTable API was not yet supported for Iceberg tables in the support case 174888831500226

On top of this behaviour, the link [1] mentions this note: **The UpdateTable API does not currently support Iceberg table format as an input to the operation.**

However, I have confirmed that the [Glue UpdateTable API](https://docs.aws.amazon.com/glue/latest/webapi/API_UpdateTable.html) is in fact compatible with Iceberg tablea. This can be in fact quickly verified by editing an Iceberg table in the AWS console and checking the corresponding UpdateTable event in CloudTrail completing successfully. However, CDK and CloudFormation have not been updated to expose this capability natively.

I want to ask for the Glue [CfnTable L1 construct in CDK](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_glue.CfnTable.html) and the related [CloudFormation components](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-glue-table.html) to allow the UpdateTable behaviour as freely as the Glue UpdateTable API and also updated the doc [1] since the note is incorrect.

Attached is a workaround [2] using [AwsCustomResource in CDK](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.custom_resources.AwsCustomResource.html), which gives full control over the Glue API and allows us to create and update Iceberg tables accordingly without any of the metadata loss issues previously encountered.

[1] - https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-iceberg.html

[2] - [glue-table-stack.js](https://github.com/user-attachments/files/26605025/glue-table-stack.js)

### AWS CDK Library version (aws-cdk-lib)

aws-cdk-lib@2.246.0

### AWS CDK CLI version

2.1115.1 (build e979d8d)

### Environment details (OS name and version, etc.)

macOS Tahoe 26.4

### Other Details

Previously approached CDK team to request this but they recommended to now approach CloudFormation team via this medium: [+] - https://github.com/aws/aws-cdk/issues/37560#issuecomment-4230667999

Contributor guide

Open the contributing guide

Research direction

Start with the AWS::Glue::Table CloudFormation resource documentation and the Glue UpdateTable API behavior described in the issue, then compare them with the CDK CfnTable L1 construct and the attached glue-table-stack.js workaround. Done means Iceberg-specific properties survive schema updates through CloudFormation/CDK and the incorrect Iceberg UpdateTable note is corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.