Enhancements to make it easier to opt out of the scan for failed items after an error in the item writer [BATCH-1511]

Sanjeevkumar opened BATCH-1511 and commented

I had a requirement of reading the file , processing it and then posting the the processed record to a table. At the same time I had to delete the data from another table. However the requirement was if the deletion fails, insertion should still happen successfully. Hence i decided to use a requires new transaction attribute for each of these method calls.

Also we had a requirement that job should never be terminated no matter what exception is thrown. At the same time all the skipped records have to be logged to the custom audit tables.

However we faced a problem when trying to call insert and delete methods from itemwriter.write() method. When the delete failed, insert would work fine. However any exception thrown in deletion would trigger the retry since we had the skip configured and would call the insert again thus resulting in duplicate transactions. The reply from the Spring Batch team was skip is a retry with limit = 0 and expected behavior.

However there might be a possibility when i would want to continue the job run irrespective of exception thrown till the threshold limit of skip is reached. However I do not want the retry to run for any record that gets skipped. Since I am auditing all the records that were skipped i do not want the job to waste the time in retrying every skipped record. Can we have a new feature in Skip where i can skip the record but i do not want the retry to happen at all.

To conclude i want to skip every record that fails and i will audit it. But i do not want the job to waste time retrying every record that fails.

Reference URL: http://forum.springsource.org/showthread.php?t=84486

Sub-tasks:

• BATCH-1518 Add optimization for commit-interval=1

• BATCH-1519 Optional skip of whole chunk

• BATCH-1520 Allow null ItemWriter as long as ItemProcessor is provided

Start with the item writer's write() flow and the configured skip and retry behavior described in the issue. Review the related sub-tasks BATCH-1518, BATCH-1519, and BATCH-1520 to understand the intended scope. Done means skipped records are audited and the job can continue without retrying each skipped record, subject to the configured skip threshold.