apache / apache/hudi

Config to Fail HoodieMultiTableStreamer Job if anyone of the table fails

Open
#16,538 2 comments 0 reactions 1 assignee Claimed by @yihua View on GitHub
area:tools from-jira priority:high type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

*strong text*We just log the table names which failed the sync. Introduce the config to fail the entire job if any of the table fails.

 

```
logger.info("Ingestion was successful for topics: " + successTables);
if (!failedTables.isEmpty()) {
logger.info("Ingestion failed for topics: " + failedTables);
}
```

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-7944
- Type: Improvement
- Fix version(s):
- 1.1.0

---

## Comments

26/Aug/24 10:45;bibhupala;I can look into this task, if this is yet not assigned.

 

Thanks,

Bibhu;;;

---

05/Dec/24 13:01;adityagoenka;Hey, Did you got a chance to make progress on this. Thanks.;;;

---

23/Dec/24 06:53;bibhupala;Hi Aditya!! 

Sorry for the delay. Was occupied with few other tasks. Have started looking into, will update by end of this week i.e. 28/12

 

Thanks;;;

---

01/Jan/25 11:50;bibhupala;Created a draft PR - [https://github.com/apache/hudi/pull/12563]

Pending items: Fixing tests and adding more tests. ;;;

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.