Large augmented diff causes infinite loop
- Dominant language
- Scala
- Stars
- 81
- Forks
- 26
- PR merge metrics
- No merged PRs in 30d
Description
We observed the following log behavior:
```
21/04/29 21:07:19 DEBUG sources.AugmentedDiffSource: Fetching sequence 4529538
21/04/29 21:07:21 ERROR sources.AugmentedDiffSource: sequence 4529538 caused an error
java.lang.OutOfMemoryError: Java heap space
```
The augmented diff was larger than usual, but the problem is that the failure mode was to retry the download infinitely. We need to allow this failure to percolate up and kill the application so that other alerts might be triggered.
Contributor guide
No contributing guide indexed for this repository
Research direction
The log identifies sources.AugmentedDiffSource and sequence 4529538; start by tracing its error and retry handling for oversized augmented diffs. Verify that this failure propagates instead of retrying indefinitely, allowing the application to terminate so external alerts can trigger.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100