MultiResourceItemWriter - Job restart failing (File is not writable) if last file first chunk failed [BATCH-2759]

Aitor Perez opened BATCH-2759 and commented

Hi,

I'm developing an Spring Batch application that writes database rows in CSV files. The database is huge so I need to generate multiple output CSV files partitioned (outputfile.csv.1, outputfile.csv.2, outputfile.csv.3 .....)

I'm testing all data cases and I have found a problem when I restart the failed job in this situation. When the execution failed in the first chunk of a new partitioned file, this new file is not created (I tried shouldDeleteIfEmpty = false but the file is not created).

When I try the restart, it seems that is expecting this last missing partitioned failed file.

Example:
ChunkSize = 50
ItemCountLimitPerResource = 100
Force Exception at item 325

The execution fails an I can see in the ouput the generated files (outputfile.csv.1, outputfile.csv.2, outputfile.csv.3) with the 300 first rows (6 chunks and 3 completed files).

When I restart the job, I'm getting the exception:

org.springframework.batch.item.ItemStreamException: File is not writable: [XXXX...\outputfile.csv.4]

It seems that expects this file, but was not created due to the error in the first chunk (I tried shouldDeleteIfEmpty = false but the file is not created)

Reviewing the database repository I can check the execution, and the value of the resource index is 4

{"map":[{"entry":[{"string":"FlatFileItemWriter.current.count","long":700},{"string":"JdbcCursorItemReader.read.count","int":300},{"string":"MultiResourceItemWriter.resource.index","int":4},{"string":"FlatFileItemWriter.written","long":50},{"string":["batch.taskletType","org.springframework.batch.core.step.item.ChunkOrientedTasklet"]},{"string":"MultiResourceItemWriter.resource.item.count","int":0},{"string":["batch.stepType","org.springframework.batch.core.step.tasklet.TaskletStep"]}]}]}

I tried configuring the FlatFileItemWriter flags with different values (AppendAllowed, ShouldDeleteIfEmpty, ShouldDeleteIfExists) but the same issue is thrown

Thanks

Affects: 4.0.1

Start with MultiResourceItemWriter restart handling and reproduce the reported case using a chunk size of 50, an item limit of 100 per resource, and a failure at item 325. Inspect the persisted resource index and writer state described in the report; done means restarting no longer fails on the missing fourth output file and the partitioned CSV output resumes correctly.