apache / apache/hudi

Create a compatibility test

Open
#17,432 0 comments 0 reactions 1 assignee Assigned to @linliu-code View on GitHub
from-jira priority:high type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Let's use this native HFile writer and HBase HFile writer to write the same key-value pairs and compare the HFile content so that they should be the same, as the final validation.

For this, we can generate a HFile using HBase-based HFile writer, add it as a test artifact ({{{}test/resources/hfile/from_hbase_writer.hfile{}}}) and add a unit/functional tests that generates a HFile using the new native HFile writer and compare the generated HFile with {{from_hbase_writer.hfile}} to make sure any change to the HFile writer that lead to storage byte change should be caught (we should also add the HFile generated by the native writer too so that byte-by-byte there is no change unless intended).

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-9455
- Type: Sub-task
- Parent: https://issues.apache.org/jira/browse/HUDI-8222
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.