[Task]: Flush Text File Output
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 476
- Avg merge
- 18h 32m
- Merged PRs (30d)
- 216
Description
### What needs to happen?
Migration: https://issues.apache.org/jira/browse/HOP-2568
Add option to flush the output stream on a regular interval or just flush every few seconds by default.
For slow-arriving data it's annoying that nothing is shown in the output file until a certain size is reached.
### Issue Priority
Priority: 3
### Issue Component
Component: Transforms
Contributor guide
Research direction
The issue does not name files or tests; start by locating the Text File Output transform in the Java source and tracing how its output stream is written and closed. Determine how a configurable interval or default periodic flush should behave for slow-arriving data, then verify that output becomes visible before the stream reaches its size threshold.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100