apache / apache/hudi

Unify configurations for update and delete during upgrade/downgrade

Open
#17,440 0 comments 0 reactions 1 assignee Assigned to @linliu-code View on GitHub
from-jira priority:critical type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

[https://github.com/apache/hudi/pull/13519#discussion_r2263549244]

During upgrade/downgrade, there are two sets of configurations: toUpdate and toRemove.

There could be some corners that a property is update/added, and then removed later, but may be added again. If we don't anything, we could see that the configuration will exist in both toUpdate and toRemove config sets.

Therefore, we need to make this logic straight.

Ethan's comment: 
{code:java}
If the table is upgraded from 6 to 9, when executing the eight-to-nine upgrade handler, based on the logic the table config fetched here is still from original version 6, because the table config is only rewritten after all upgrade is called. Does the logic here assume that some table configs are already updated to table version 8? I think this assumption make senses for maintaining the upgrade logic, but it requires changing the logic to follow that, i.e., passing in the updated and deleted configs from 6->8 upgrades. {code}
 

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-9702
- Type: Sub-task
- Parent: https://issues.apache.org/jira/browse/HUDI-9560
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.