Fix marker based strategy for structured streaming
Open
from-jira
priority:critical
type:bug
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
As part of [this|https://github.com/apache/hudi/pull/3967] patch, we are making timeline server based as the default marker type. But we have an issue w/ structured streaming. Looks like after 1st micro batch, the timeline server gets shutdown and for subsequent micro batches, timeline server is not available. So, in the patch we have made marker based overridden just for structured streaming.
We may want to revisit this and see how to go about it.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-2782
- Type: Bug
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.