aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap

Glue Iceberg Table: Table is broken after any update

Open
#1,919 34 comments 41 reactions 0 assignees View on GitHub
bug
Dominant language
No language data
Stars
1.1k
Forks
62
PR merge metrics
No merged PRs in 30d

Description

### Name of the resource

AWS::Glue::Table

### Resource Name

_No response_

### Issue Description

Hi there!
When I try to update something on my iceberg table, the update causes the table to break and the table format to disappear. Basically, it's no longer an iceberg table and no operations on the table are possible.

### Expected Behavior

When I update the table, the update does not remove the table input and I can work with the iceberg table as I should.

### Observed Behavior

Before the update (after initial deployment):
image

After any update:
image

Notice the table format prop. Table management prop is also away.

Athena before update:
![Zrzut ekranu 2024-02-2 o 14 23 06](https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/assets/30108423/864506fe-db5d-4179-8dbf-2d5c4c4be2c0)

Athena after update:
![image](https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/assets/30108423/3004bbe2-7a72-434e-babe-e031576f3d7a)
![image](https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/assets/30108423/0ff3d20c-c8a9-4468-aca1-804feecdbc01)

### Test Cases

Simple CDK Stack to reproduce this behavior (uncomment one column to update, or do any other update):
```typescript
export class CdkTestingStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);

const myTestDatabase = new CfnDatabase(this, 'myTestDatabase', {
catalogId: Aws.ACCOUNT_ID,
databaseInput: {
name: 'mytestdatabase'
}
})

const myLocationBucket = new Bucket(this, 'myLocationBucket', {
removalPolicy: RemovalPolicy.DESTROY,
autoDeleteObjects: true
})

const myTestTable = new CfnTable(this, 'myTestTable', {
databaseName: 'mytestdatabase',
catalogId: Aws.ACCOUNT_ID,
tableInput: {
name: 'mytesttable',
storageDescriptor: {
columns: [
{
name: 'name',
type: 'string'
},
// {
// name: 'ts',
// type: 'timestamp'
// }
],
location: `s3://${myLocationBucket.bucketName}/mytesttable/`,
},
tableType: 'EXTERNAL_TABLE',
},
openTableFormatInput: {
icebergInput: {
metadataOperation: 'CREATE',
version: '2'
}
}
})

}
}
```

### Other Details

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the TypeScript CDK reproduction in the issue, focusing on AWS::Glue::Table's tableInput and openTableFormatInput during an update. Deploy the example, make the shown column change, and inspect the Glue and Athena table state before and after; done means updates preserve the Iceberg table format and table-management properties.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.