apache / apache/hudi

Improve the format of record merge strategy ID

Open
#17,297 1 comment 0 reactions 1 assignee Assigned to @yihua View on GitHub
from-jira priority:critical type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

[https://github.com/apache/hudi/pull/11943#discussion_r1809697340]

Could we make the ID easier to read, e.g.,
"00000000-0000-0000-0000-000000000000": Avro payload based merging;
"00000000-0000-0000-0000-000000000001": overwrite with latest;
"00000000-0000-0000-0000-000000000002": event time ordering / default hoodie payload. Still reserve the existing ID as an alternative ID;
"00000000-0000-0000-0000-000000000003": validate duplicate key record merger;
etc.

Also is it necessary to use UUID format, maybe "0", "1", "2" are shorter IDs?

Another problem is, In the current scheme, it's still hard to avoid collision, since there is no rule on how to generate these IDs. A general String with namespace and number might be better.

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-8422
- Type: Sub-task
- Parent: https://issues.apache.org/jira/browse/HUDI-9108
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.