apache / apache/hudi

Improve readability and maintainability of metadata table (MT/MDT) write path

Open
#16,903 1 comment 0 reactions 1 assignee Claimed by @yihua View on GitHub
from-jira priority:blocker type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

The core logic of metadata table write path sits in these classes:
{code:java}
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter
org.apache.hudi.metadata.SparkHoodieBackedTableMetadataWriter
org.apache.hudi.client.utils.SparkMetadataWriterUtils
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter
org.apache.hudi.metadata.SecondaryIndexRecordGenerationUtils {code}
The readability and maintainability need to be improved by providing better abstractions.  The goal is to get it to a point where a new index type can be added easily.

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-9176
- Type: Improvement
- Epic: https://issues.apache.org/jira/browse/HUDI-7537
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.