Check shutdown logic with async compaction in Spark Structured Streaming
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Related issue
https://github.com/apache/hudi/issues/5046
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3467
- Type: Bug
- Epic: https://issues.apache.org/jira/browse/HUDI-4388
- Fix version(s):
- 1.1.0
---
## Comments
28/Feb/22 20:04;shivnarayan;Update:
I confirmed that when regular writes fail, async table services shuts down. and when async table services shuts down, regular writes fail.
But just that I could not exit from the spark-shell. It is just stuck. I had to use `kill -9 pid` to kill the process altogether. But chances that spark-streaming works that way. since reading from df stream is continuous.
This is not an immediate necessity to fix, since both of them gets shutdown on any issue.
;;;
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.