spring-projects / spring-projects/spring-batch
Incorrect handling of consecutive faulty items in chunk scanning
Open
Nobody has claimed this yet.
has: minimal-example
in: core
related-to: fault-tolerance
type: bug
- Dominant language
- Java
- Stars
- 3k
- Forks
- 2.5k
- Avg merge
- 6d 53m
- Merged PRs (30d)
- 3
Description
As of v5.0.1, the FaultTolerantChunkProcessor is unable to skip two consecutive faulty items when scanning chunks.
Here is a failing test (currently disabled in FaultTolerantChunkProcessorTests):
@Test
void testWriteRetryOnTwoExceptions() throws Exception {
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
retryPolicy.setMaxAttempts(2);
batchRetryTemplate.setRetryPolicy(retryPolicy);
processor.setWriteSkipPolicy(new AlwaysSkipItemSkipPolicy());
processor.setItemWriter(new ItemWriter<String>() {
@Override
public void write(Chunk<? extends String> chunk) throws Exception {
if (chunk.getItems().contains("fail")) {
throw new IllegalArgumentException("Expected Exception!");
}
}
});
Chunk<String> inputs = new Chunk<>(Arrays.asList("3", "fail", "fail", "4"));
Exception exception = assertThrows(RuntimeException.class, () -> processor.process(contribution, inputs));
assertEquals("Expected Exception!", exception.getMessage());
// first retry
exception = assertThrows(RuntimeException.class, () -> processor.process(contribution, inputs));
assertEquals("Expected Exception!", exception.getMessage());
// retry exhausted, now scanning
processor.process(contribution, inputs);
// skip on this attempt
exception = assertThrows(RuntimeException.class, () -> processor.process(contribution, inputs));
assertEquals("Expected Exception!", exception.getMessage());
// 2nd exception detected
exception = assertThrows(RuntimeException.class, () -> processor.process(contribution, inputs));
assertEquals("Expected Exception!", exception.getMessage());
// still scanning
processor.process(contribution, inputs);
assertEquals(2, contribution.getSkipCount());
assertEquals(2, contribution.getWriteCount());
assertEquals(0, contribution.getFilterCount());
}
This test started failing after resolving #4314.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by re-enabling the failing test in FaultTolerantChunkProcessorTests and reading the FaultTolerantChunkProcessor flow it exercises. Run the test to reproduce the consecutive-fault case; done means the test passes and the final assertions report two skipped items, two writes, and no filtered items.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100