apache / apache/hudi

Fix all complete instant fetch from timeline in test

Open
#16,867 1 comment 0 reactions 1 assignee Claimed by @yihua View on GitHub
from-jira priority:high type:bug
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

As of today, when a test would like to read completed instant file, it typcially does:
* use request time string as a key to fetch
* create a dummy instant who only comes with request time yet no completion time.

This is fine in old timeline layout version, yet it no longer works in new timeline version. We would end up with "file not found" since the instant file name generation based on the dummy instant does not exists in FS at all.

 

So today the workaround is in the implementation code we fix this instant object on the fly.

In future we should remove this logic and fix all the tests
{code:java}
@Override
public Option getContentStream(HoodieInstant instant) {
Option actualInstant = Option.of(instant);

if (instant.getCompletionTime().isEmpty() && instant.getState().equals(HoodieInstant.State.COMPLETED)) {
actualInstant = getInstantFromTimeline(instant, this, actualInstant);
}
if (actualInstant.isEmpty()) {
return Option.empty();
}
StoragePath filePath = getInstantFileNamePath(instantFileNameGenerator.getFileName(actualInstant.get()));
return Option.of(readDataStreamFromPath(filePath));
} {code}

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-9063
- Type: Bug
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.