apache / apache/hudi

Incorrect Record Key Field property Handling

Open
#15,021 0 comments 0 reactions 1 assignee Claimed by @yihua View on GitHub
from-jira priority:blocker type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Currently `HoodieTableConfig#getRecordKeyFieldProp` returns a single String, even though it could contain a *list* of columns making up composite Primary Key.
{code:java}
public String getRecordKeyFieldProp() {
return getStringOrDefault(RECORDKEY_FIELDS, HoodieRecord.RECORD_KEY_METADATA_FIELD);
} {code}
 

Most of the callers of this method are actually not handling this correctly, assuming that the Record Key is always a single field. 

NOTE: While concatenation of CPK seems like a very natural step here, special care has to be taken, since Composite PK can NOT be concatenated as strings, as this might break the uniqueness constraint. 

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-3411
- Type: Improvement
- Fix version(s):
- 1.1.0

---

## Comments

22/Feb/22 06:34;todd5167;in addition,KeyGenUtils#getRecordKey(org.apache.avro.generic.GenericRecord, java.lang.String, boolean) and KeyGenUtils#getRecordKey(org.apache.avro.generic.GenericRecord, java.lang.String, boolean)

There are some differences in the format of the generated key, which causes the recordkey and the recordkey written by flink to not match correctly.

If you want to modify, how to unify.;;;

---

16/Mar/22 00:56;xushiyan;as discussed with [~alexey.kudinkin] , we need to evaluate the risk of skipping the proper fix.;;;

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.