Reduce merge info logging.
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 487
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 13
Description
The current merge info that is printed to the logs is spammy, but may require significant changes to reduce the logging while still providing some useful information to monitor / trouble shooting merge operations. The core issue is that each time the next merge state is recalculated the merge overlap information is logged.
One way may be to print the overlaps on creation and on completion, but suppress or summarize the intermediate states.
The current logging (for a small merge) for each merge calculation looks like:
```
2022-06-28T14:21:58,034 [manager.Manager] DEBUG: Creating merge op: 3 from startRow: -inf to endRow: +inf
2022-06-28T14:21:58,175 [state.MergeInfo] DEBUG: mergeInfo 3;g< overlaps: 3<<
2022-06-28T14:21:58,176 [state.MergeInfo] DEBUG: mergeInfo 3;k;g overlaps: 3<<
2022-06-28T14:21:58,176 [state.MergeInfo] DEBUG: mergeInfo 3;m;k overlaps: 3<<
2022-06-28T14:21:58,176 [state.MergeInfo] DEBUG: mergeInfo 3<;m overlaps: 3<<
```
Click to expand for full merge operation example
```
2022-06-28T14:21:58,034 [manager.Manager] DEBUG: Creating merge op: 3 from startRow: -inf to endRow: +inf
2022-06-28T14:21:58,175 [state.MergeInfo] DEBUG: mergeInfo 3;g< overlaps: 3<<
2022-06-28T14:21:58,176 [state.MergeInfo] DEBUG: mergeInfo 3;k;g overlaps: 3<<
2022-06-28T14:21:58,176 [state.MergeInfo] DEBUG: mergeInfo 3;m;k overlaps: 3<<
2022-06-28T14:21:58,176 [state.MergeInfo] DEBUG: mergeInfo 3<;m overlaps: 3<<
2022-06-28T14:21:58,176 [state.MergeStats] INFO : Computing next merge state for 3<< which is presently STARTED isDelete : false
2022-06-28T14:21:58,288 [state.MergeInfo] DEBUG: mergeInfo 3;g< overlaps: 3<<
2022-06-28T14:21:58,290 [state.MergeInfo] DEBUG: mergeInfo 3;k;g overlaps: 3<<
2022-06-28T14:21:58,290 [state.MergeInfo] DEBUG: mergeInfo 3;m;k overlaps: 3<<
2022-06-28T14:21:58,291 [state.MergeInfo] DEBUG: mergeInfo 3<;m overlaps: 3<<
2022-06-28T14:21:58,292 [state.MergeStats] INFO : Computing next merge state for 3<< which is presently WAITING_FOR_CHOPPED isDelete : false
2022-06-28T14:21:58,908 [state.MergeInfo] DEBUG: mergeInfo 3;g< overlaps: 3<<
2022-06-28T14:21:58,911 [state.MergeInfo] DEBUG: mergeInfo 3;k;g overlaps: 3<<
2022-06-28T14:21:58,911 [state.MergeInfo] DEBUG: mergeInfo 3;m;k overlaps: 3<<
2022-06-28T14:21:58,912 [state.MergeInfo] DEBUG: mergeInfo 3<;m overlaps: 3<<
2022-06-28T14:21:58,912 [state.MergeStats] INFO : Computing next merge state for 3<< which is presently WAITING_FOR_CHOPPED isDelete : false
2022-06-28T14:21:59,021 [state.MergeInfo] DEBUG: mergeInfo 3;g< overlaps: 3<<
2022-06-28T14:21:59,021 [state.MergeInfo] DEBUG: mergeInfo 3;k;g overlaps: 3<<
2022-06-28T14:21:59,022 [state.MergeInfo] DEBUG: mergeInfo 3;m;k overlaps: 3<<
2022-06-28T14:21:59,022 [state.MergeInfo] DEBUG: mergeInfo 3<;m overlaps: 3<<
2022-06-28T14:21:59,022 [state.MergeStats] INFO : Computing next merge state for 3<< which is presently WAITING_FOR_OFFLINE isDelete : false
2022-06-28T14:21:59,163 [state.MergeInfo] DEBUG: mergeInfo 3<< overlaps: 3<<
2022-06-28T14:21:59,166 [state.MergeStats] INFO : Computing next merge state for 3<< which is presently COMPLETE isDelete : false
2022-06-28T14:21:59,178 [merge.TableRangeOpWait] INFO : removing merge information No Merge in progress
```
Contributor guide
Research direction
Start by tracing the merge logging shown from state.MergeInfo, state.MergeStats, manager.Manager, and merge.TableRangeOpWait. Review when overlap information is emitted during merge-state recalculation, then define a focused logging change that preserves useful creation and completion information while suppressing or summarizing intermediate output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100