aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap
[AWS::Glue::TableOptimizer] - [Docs] - Properties are completely incorrect
- Dominant language
- No language data
- Stars
- 1.1k
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
### Name of the resource
AWS::Glue::TableOptimizer
### Resource name
_No response_
### Reference Link
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-tableoptimizer.html
### Details
1. Type:
```
The type of table optimizer. Currently, the only valid value is compaction.
Required: Yes
Type: String
```
It's wrong, three values are valid:
- `compaction`
- `retention`
- `orphan_file_deletion`
CLI Reference: https://docs.aws.amazon.com/cli/latest/reference/glue/create-table-optimizer.html
I've been able to use all three values successfully and get the expected result.
2. TableOptimizerConfiguration.RetentionConfiguration.IcebergConfiguration:
```
Location: String
OrphanFileRetentionPeriodInDays: Integer
```
It's not. Actually correct structure is:
```
SnapshotRetentionPeriodInDays: Integer
NumberOfSnapshotsToRetain: Integer
CleanExpiredFiles: Boolean
```
Also CLI as reference. Also tested.
3. TableOptimizerConfiguration based on Type
- If `Type` is `compaction`, you cannot specify either `OrphanFileDeletionConfiguration` or `RetentionConfiguration`
- If `Type` is `retention`, you have to provide `RetentionConfiguration`
- If `Type` is `orphan_file_deletion`, you have to provide `OrphanFileDeletionConfiguration`
The CDK implementation of CFN resources is also poor due to incorrect properties, but I have a workaround for folks who want to automate TableOptimizer:
```typescript
const cfnTableOptimizerCompaction = new CfnTableOptimizer(testStack, 'TableOptimizerCompaction', {
catalogId: catalogId, //essentially AWS Account Id
databaseName: dbName,
tableName: tableName,
type: 'compaction',
tableOptimizerConfiguration: {
enabled: true,
//correct permissions here: https://docs.aws.amazon.com/glue/latest/dg/optimization-prerequisites.html
roleArn: compactionRoleArn
}
});
const cfnTableOptimizerOrphanFileDeletion = new CfnTableOptimizer(testStack, 'TableOptimizerOrphanFileDeletion', {
catalogId: catalogId,
databaseName: dbName,
tableName: tableName,
type: 'orphan_file_deletion',
tableOptimizerConfiguration: {
orphanFileDeletionConfiguration: {
icebergConfiguration: {
// ex: 's3://some-bucket/table', can be found in Table Details in Glue Console
location: tableLocation,
orphanFileRetentionPeriodInDays: 1
}
},
enabled: true,
roleArn: orphanFileDeletionRoleArn
}
});
const cfnTableOptimizerRetention = new CfnTableOptimizer(testStack, 'TableOptimizerRetention', {
catalogId: catalogId,
databaseName: dbName,
tableName: tableName,
type: 'retention',
tableOptimizerConfiguration: {
enabled: true,
roleArn: retentionRoleArn
}
});
cfnTableOptimizerRetention.addOverride('Properties.TableOptimizerConfiguration.RetentionConfiguration.IcebergConfiguration', {
SnapshotRetentionPeriodInDays: 1,
NumberOfSnapshotsToRetain: 1,
CleanExpiredFiles: true
})
```
Contributor guide
Research direction
Start with the AWS::Glue::TableOptimizer reference link and the AWS CLI create-table-optimizer reference cited in the issue. Verify the three Type values, the RetentionConfiguration.IcebergConfiguration fields, and the Type-dependent configuration requirements; done means the CloudFormation documentation matches the tested AWS behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100