Incorrect Record Key Field property Handling
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Currently `HoodieTableConfig#getRecordKeyFieldProp` returns a single String, even though it could contain a *list* of columns making up composite Primary Key.
{code:java}
public String getRecordKeyFieldProp() {
return getStringOrDefault(RECORDKEY_FIELDS, HoodieRecord.RECORD_KEY_METADATA_FIELD);
} {code}
Most of the callers of this method are actually not handling this correctly, assuming that the Record Key is always a single field.
NOTE: While concatenation of CPK seems like a very natural step here, special care has to be taken, since Composite PK can NOT be concatenated as strings, as this might break the uniqueness constraint.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3411
- Type: Improvement
- Fix version(s):
- 1.1.0
---
## Comments
22/Feb/22 06:34;todd5167;in addition,KeyGenUtils#getRecordKey(org.apache.avro.generic.GenericRecord, java.lang.String, boolean) and KeyGenUtils#getRecordKey(org.apache.avro.generic.GenericRecord, java.lang.String, boolean)
There are some differences in the format of the generated key, which causes the recordkey and the recordkey written by flink to not match correctly.
If you want to modify, how to unify.;;;
---
16/Mar/22 00:56;xushiyan;as discussed with [~alexey.kudinkin] , we need to evaluate the risk of skipping the proper fix.;;;
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.