DATAJPA-931 breaks merging with RepositoryItemWriter [BATCH-2678]

Derek Lewis opened BATCH-2678 and commented

DATAJPA-931 introduced an optimization that guards against JPA entity merging when the entity is still attached. This can impact Spring Batch jobs, where the following happens:

• 1 Job defined w/ the following step:
  • One step reads using rows from table A using JdbcPagingItemReader and creates entity A's to write via RepositoryItemWriter
• Another job defined w/ the following step (assume this job is shortly called after the 1st one):
  • A subsequent step reads the same rows from table A (potentially with additional columns now being read) using JdbcPagingItemReader and creates additional entity A's to write via RepositoryItemWriter. Prior to this Spring Data JPA optimization, the two entity A's would be merged and a SQL update would be performed.
• Entity A's repository is defined as a PagingAndSortingRepository.

This is what happens on the 2nd job w/ Spring Data Kay SR3 (/w optimization) and Spring Boot 4.0.0.RELEASE (.M5 is in the output below, but it happens with both releases):

org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [a_encounter_pkey]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:259) ~[spring-orm-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:225) ~[spring-orm-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:540) ~[spring-orm-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:742) ~[spring-tx-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:710) ~[spring-tx-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at jdk.internal.reflect.GeneratedMethodAccessor125.invoke(Unknown Source) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:564) ~[na:na]
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338) [spring-aop-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) [spring-aop-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at com.sun.proxy.$Proxy155.commit(Unknown Source) ~[na:na]
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:152) ~[spring-tx-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:272) ~[spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81) ~[spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375) ~[spring-batch-infrastructure-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145) ~[spring-batch-infrastructure-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257) ~[spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:394) [spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]
	at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.3.RELEASE.jar!/:5.0.3.RELEASE]
	at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.0.M5.jar!/:4.0.0.M5]

I was able to get the same code working again by changing entity A repository to a custom JpaRepository implementation that can do a save/detach of the entity on save and updating RepositoryItemWriter to utilize that custom method.

Also, everything works fine when I use release Kay SR2 and prior releases of Spring Data.

Affects: 4.0.0

Reference URL: https://jira.spring.io/browse/DATAJPA-931

Issue Links:

• DATAJPA-1261 Revert optimizations made for existing entities in implementation of CrudRepository.save(…)

Start with RepositoryItemWriter and the DATAJPA-931 and DATAJPA-1261 references; reproduce the two sequential jobs described in the issue, where JdbcPagingItemReader creates entities for the same rows. Done means the second job merges existing entities and performs an update rather than failing with a duplicate primary-key constraint.