aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap

[AWS::Glue::TableOptimizer] - [Docs] - Properties are completely incorrect

Open
#2,257 1 comment 7 reactions 0 assignees View on GitHub
documentation
Dominant language
No language data
Stars
1.1k
Forks
62
PR merge metrics
No merged PRs in 30d

Description

### Name of the resource

AWS::Glue::TableOptimizer

### Resource name

_No response_

### Reference Link

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-tableoptimizer.html

### Details

1. Type:
```
The type of table optimizer. Currently, the only valid value is compaction.

Required: Yes

Type: String
```

It's wrong, three values are valid:

- `compaction`
- `retention`
- `orphan_file_deletion`

CLI Reference: https://docs.aws.amazon.com/cli/latest/reference/glue/create-table-optimizer.html
I've been able to use all three values successfully and get the expected result.

2. TableOptimizerConfiguration.RetentionConfiguration.IcebergConfiguration:
```
Location: String
OrphanFileRetentionPeriodInDays: Integer
```

It's not. Actually correct structure is:
```
SnapshotRetentionPeriodInDays: Integer
NumberOfSnapshotsToRetain: Integer
CleanExpiredFiles: Boolean
```

Also CLI as reference. Also tested.

3. TableOptimizerConfiguration based on Type

- If `Type` is `compaction`, you cannot specify either `OrphanFileDeletionConfiguration` or `RetentionConfiguration`
- If `Type` is `retention`, you have to provide `RetentionConfiguration`
- If `Type` is `orphan_file_deletion`, you have to provide `OrphanFileDeletionConfiguration`

The CDK implementation of CFN resources is also poor due to incorrect properties, but I have a workaround for folks who want to automate TableOptimizer:

```typescript
const cfnTableOptimizerCompaction = new CfnTableOptimizer(testStack, 'TableOptimizerCompaction', {
catalogId: catalogId, //essentially AWS Account Id
databaseName: dbName,
tableName: tableName,
type: 'compaction',
tableOptimizerConfiguration: {
enabled: true,
//correct permissions here: https://docs.aws.amazon.com/glue/latest/dg/optimization-prerequisites.html
roleArn: compactionRoleArn
}
});
const cfnTableOptimizerOrphanFileDeletion = new CfnTableOptimizer(testStack, 'TableOptimizerOrphanFileDeletion', {
catalogId: catalogId,
databaseName: dbName,
tableName: tableName,
type: 'orphan_file_deletion',
tableOptimizerConfiguration: {
orphanFileDeletionConfiguration: {
icebergConfiguration: {
// ex: 's3://some-bucket/table', can be found in Table Details in Glue Console
location: tableLocation,
orphanFileRetentionPeriodInDays: 1
}
},
enabled: true,
roleArn: orphanFileDeletionRoleArn
}
});
const cfnTableOptimizerRetention = new CfnTableOptimizer(testStack, 'TableOptimizerRetention', {
catalogId: catalogId,
databaseName: dbName,
tableName: tableName,
type: 'retention',
tableOptimizerConfiguration: {
enabled: true,
roleArn: retentionRoleArn
}
});
cfnTableOptimizerRetention.addOverride('Properties.TableOptimizerConfiguration.RetentionConfiguration.IcebergConfiguration', {
SnapshotRetentionPeriodInDays: 1,
NumberOfSnapshotsToRetain: 1,
CleanExpiredFiles: true
})
```

Contributor guide

Open the contributing guide

Research direction

Start with the AWS::Glue::TableOptimizer reference link and the AWS CLI create-table-optimizer reference cited in the issue. Verify the three Type values, the RetentionConfiguration.IcebergConfiguration fields, and the Type-dependent configuration requirements; done means the CloudFormation documentation matches the tested AWS behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.