[SUPPORT] AWSGlueCatalogSyncClient.updateTableComments applies no comments since AWS SDK v2 upgrade
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
**Describe the problem you faced**
`AWSGlueCatalogSyncClient.updateTableComments` no longer applies any column or partition column comments since the AWS SDK v2 upgrade.
Its helper `setComments` builds a new `Column` and discards the result instead of replacing the element in the list:
```java
private void setComments(List columns, Map> commentsMap) {
columns.forEach(column -> {
String comment = commentsMap.getOrDefault(column.name(), Option.empty()).orElse(null);
Column.builder().comment(comment).build(); // result dropped, column is unchanged
});
}
```
AWS SDK v2 model classes are immutable, so mutating in place is not possible. Before the SDK v2 upgrade (e9dd73fc08fd, #9347) this code called `column.setComment(...)` on the mutable v1 model, which worked. As a result `updateTableComments` never detects a change and always returns `false`, and no comments are synced to Glue on the update path.
The existing Glue comment tests do not catch this because they only assert comments that were pre-baked into the mocked `Column` objects.
**Expected behavior**
With `hoodie.datasource.hive_sync.sync_comment=true`, column and partition column comments should be applied to the Glue table.
**Possible fix**
Rebuild the column lists with the comment applied, e.g.:
```java
for (int i = 0; i < columns.size(); i++) {
Column column = columns.get(i);
String comment = commentsMap.getOrDefault(column.name(), Option.empty()).orElse(null);
columns.set(i, column.toBuilder().comment(comment).build());
}
```
(using mutable copies of `storageDescriptor.columns()` / `table.partitionKeys()`, which are returned as unmodifiable lists by the SDK), and update the tests to assert comments that were not already present on the input columns.
**Environment Description**
* Hudi version: master (1.3.0-SNAPSHOT), broken since the AWS SDK v2 upgrade (#9347)
**Additional context**
Found during review of #19289, which fixes the equivalent HMS paths. Related to HUDI-8843 / #17359.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.