apache / apache/hudi

Code cleanup: HFileDataBock - using integer keys is never used

Open
#14,942 0 comments 0 reactions 1 assignee Claimed by @ad1happy2go View on GitHub
area:code-quality area:metadata-table from-jira priority:medium type:feature
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

KeyField can never be empty for File. If so, there is really no need for falling back to sequential integer keys in the HFileDataBlock::serializeRecords() code path.

 
{noformat}
// Build the record key
final Field schemaKeyField = records.get(0).getSchema().getField(this.keyField);
if (schemaKeyField == null) {
// Missing key metadata field. Use an integer sequence key instead.
useIntegerKey = true;
keySize = (int) Math.ceil(Math.log(records.size())) + 1;
}

while (itr.hasNext()) {
IndexedRecord record = itr.next();
String recordKey;
if (useIntegerKey) {
recordKey = String.format("%" + keySize + "s", key++);
} else {
recordKey = record.get(schemaKeyField.pos()).toString();
}

{noformat}

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-2954
- Type: Wish
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.