Changed behaviour with Jobstatus in Spring Batch 6 with JobExecutionListener

We use Spring Batch with an external scheduler (Control-M) and start our jobs via a REST API, which internally launches the job using the JobLauncher. Through this API, the JobRepository is periodically queried via the JobExplorer during execution in order to determine the job status. Since Spring Batch 6, this no longer works in the same way as in previous versions.

The reason for this behavior is the following commit:
https://github.com/spring-projects/spring-batch/commit/36068b5db84ff242032e9b00515454a84e0745d2

With this change, the job status is now written to the JobRepository before the afterJob method of the JobExecutionListener is invoked. This means that the scheduler may recognize the job as finished even though the code in the afterJob method is still being executed. This behavior is particularly hard to understand, since the job status can still be modified within the afterJob method.

As an example, we use the JobExecutionListener to perform reporting for the job. These reports must then be transferred to an archive system, which the scheduler is supposed to trigger as the next step after the job has finished. However, this reporting step can fail, in which case the overall job must be set to FAILED so that the scheduler does not trigger the next step.

Is this changed behavior intended, or is it a bug? With this implementation there is now no longer any way from outside to query via the JobRepository when the job has truly finished (i.e. including the execution of the afterJob method).

Start with commit 36068b5db84ff242032e9b00515454a84e0745d2 and trace the JobLauncher, JobRepository, JobExplorer, and JobExecutionListener lifecycle described in the issue. Reproduce a job whose afterJob method changes its status, then establish whether repository completion should include afterJob and capture the agreed behavior in a regression test or documentation.