Reduce log noise coming from File sink
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
After switching to Beam 2.2 I noticed that File sink related classes generate lots of lines likes this:
```
2018-01-16 01:37:37,080 INFO org.apache.beam.sdk.io.FileBasedSink - No output files to write.
2018-01-16
01:37:37,104 INFO org.apache.beam.sdk.io.WriteFiles - Will finalize 0 files
```
I did some counts and it looks like this lines account for 82% of all lines in the log yet not giving much information.
I am happy to raise a PR to make "No output files to write." and "Will finalize {} files" logged at DEBUG level.
Imported from Jira [BEAM-3488](https://issues.apache.org/jira/browse/BEAM-3488). Original Jira may contain additional context.
Reported by: pawelbartoszek.
Contributor guide
Research direction
Locate the logging statements in the FileBasedSink and WriteFiles classes that emit "No output files to write." and "Will finalize {} files". Confirm the desired behavior is for both messages to use DEBUG logging, then verify that the File sink no longer produces these messages at INFO level.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100