Help!org.springframework.orm.jpa.JpaSystemException: Unable to commit against JDBC Connection

• I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

我使用的是seata的AT模式和file模式
通过spring data jpa偶尔会得到以下异常 org.springframework.orm.jpa.JpaSystemException: Unable to commit against JDBC Connection; nested exception is org.hibernate.TransactionException: Unable to commit against JDBC Connection

Ⅱ. Describe what happened

如你所见,我的java app异常log和seata server的异常log都在这里了

在抛出这个异常之前,本应用只发了一条insert语句和一些查询语句,insert语句就是保存合同历史的insert sql

1. 我的java app异常信息是这样的
   `
   2020-04-23 14:02:21.692 ERROR --- [http-nio-22000-exec-9] c.p.c.s.ContractService : 保存合同历史失败---org.springframework.orm.jpa.JpaSystemException: Unable to commit against JDBC Connection; nested exception is org.hibernate.TransactionException: Unable to commit against JDBC Connection
   at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:351)
   at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:253)
   at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:536)
   at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:746)
   at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714)
   at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:534)
   at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:305)
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:138)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:61)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
   at com.sun.proxy.$Proxy240.save(Unknown Source)
   at com.prnasia.contract.service.ContractService.saveHistory(ContractService.java:96)
   at com.prnasia.contract.service.sf_sync.AContractSFSyncService.syncUpdate(AContractSFSyncService.java:145)
   at com.prnasia.contract.service.sf_sync.AContractSFSyncService.sync(AContractSFSyncService.java:92)
   at com.prnasia.contract.service.sf_sync.AContractSFSyncService$$FastClassBySpringCGLIB$$a019474b.invoke()
   at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
   at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
   at io.seata.spring.annotation.GlobalTransactionalInterceptor$1.execute(GlobalTransactionalInterceptor.java:104)
   at io.seata.tm.api.TransactionalTemplate.execute(TransactionalTemplate.java:64)
   at io.seata.spring.annotation.GlobalTransactionalInterceptor.handleGlobalTransaction(GlobalTransactionalInterceptor.java:101)
   at io.seata.spring.annotation.GlobalTransactionalInterceptor.invoke(GlobalTransactionalInterceptor.java:76)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
   at com.prnasia.contract.service.sf_sync.CloseContractSFSyncService$$EnhancerBySpringCGLIB$$800da492.sync()
   at com.prnasia.contract.controller.ContractSalesforceController.sync(ContractSalesforceController.java:66)
   at com.prnasia.contract.controller.ContractSalesforceController.syncNew(ContractSalesforceController.java:133)
   at sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
   at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
   at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
   at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892)
   at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797)
   at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
   at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1039)
   at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
   at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
   at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
   at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:88)
   at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
   at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92)
   at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
   at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.springframework.cloud.sleuth.instrument.web.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:50)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at brave.servlet.TracingFilter.doFilter(TracingFilter.java:99)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114)
   at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104)
   at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
   at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
   at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
   at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
   at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
   at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:853)
   at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587)
   at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.lang.Thread.run(Thread.java:748)
   Caused by: org.hibernate.TransactionException: Unable to commit against JDBC Connection
   at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:87)
   at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:272)
   at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
   at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:532)
   ... 100 more
   Caused by: java.sql.SQLException: io.seata.core.exception.TransactionException: Response[TransactionException[Could not found global transaction xid = 192.168.10.11:8091:2031428229]]
   at io.seata.rm.datasource.ConnectionProxy.recognizeLockKeyConflictException(ConnectionProxy.java:135)
   at io.seata.rm.datasource.ConnectionProxy.processGlobalTransactionCommit(ConnectionProxy.java:184)
   at io.seata.rm.datasource.ConnectionProxy.commit(ConnectionProxy.java:161)
   at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:81)
   ... 103 more
   Caused by: io.seata.core.exception.TransactionException: Response[TransactionException[Could not found global transaction xid = 192.168.10.11:8091:2031428229]]
   at io.seata.rm.AbstractResourceManager.branchRegister(AbstractResourceManager.java:68)
   at io.seata.rm.DefaultResourceManager.branchRegister(DefaultResourceManager.java:96)
   at io.seata.rm.datasource.ConnectionProxy.register(ConnectionProxy.java:207)
   at io.seata.rm.datasource.ConnectionProxy.processGlobalTransactionCommit(ConnectionProxy.java:182)
   ... 105 more

2020-04-23 14:02:21.693 ERROR --- [http-nio-22000-exec-9] c.p.c.s.s.AContractSFSyncService : SF-DS更新合同失败CTPKHK2004002
`

2. 在seata server的log是这样的
   2020-04-23 14:00:29.676 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.lambda$branchRegister$0:98 -Successfully register branch xid = 192.168.10.11:8091:2031428229, branchId = 2031429560 2020-04-23 14:00:29.686 INFO [batchLoggerPrint_1]io.seata.core.rpc.DefaultServerMessageListenerImpl.run:205 -SeataMergeMessage xid=192.168.10.11:8091:2031428229,branchId=2031429560,resourceId=null,status=PhaseOne_Done,applicationData=null ,clientIp:192.168.10.11,vgroup:ds-resource-service-seata-service-group 2020-04-23 14:00:29.686 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.branchReport:126 -Successfully branch report xid = 192.168.10.11:8091:2031428229, branchId = 2031429560 2020-04-23 14:00:29.703 INFO [batchLoggerPrint_1]io.seata.core.rpc.DefaultServerMessageListenerImpl.run:205 -SeataMergeMessage xid=192.168.10.11:8091:2031428229,branchType=AT,resourceId=jdbc:mysql://${IP}:${port}/data_sync,lockKey=ds_orderresourcepool:5157519683c640a4a51081e1f44890e3;ds_orderresourcepool:f5ced1a85f874a17ab69ede8c031faea ,clientIp:192.168.10.11,vgroup:ds-resource-service-seata-service-group 2020-04-23 14:00:29.703 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.lambda$branchRegister$0:98 -Successfully register branch xid = 192.168.10.11:8091:2031428229, branchId = 2031429561 2020-04-23 14:00:29.718 INFO [batchLoggerPrint_1]io.seata.core.rpc.DefaultServerMessageListenerImpl.run:205 -SeataMergeMessage xid=192.168.10.11:8091:2031428229,branchId=2031429561,resourceId=null,status=PhaseOne_Done,applicationData=null ,clientIp:192.168.10.11,vgroup:ds-resource-service-seata-service-group 2020-04-23 14:00:29.718 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.branchReport:126 -Successfully branch report xid = 192.168.10.11:8091:2031428229, branchId = 2031429561 2020-04-23 14:00:29.725 INFO [batchLoggerPrint_1]io.seata.core.rpc.DefaultServerMessageListenerImpl.run:205 -SeataMergeMessage xid=192.168.10.11:8091:2031428229,extraData=null ,clientIp:127.0.0.1,vgroup:salesforce-ds-integration-app-fescar-service-group 2020-04-23 14:00:29.786 INFO [batchLoggerPrint_1]io.seata.core.rpc.DefaultServerMessageListenerImpl.run:205 -SeataMergeMessage xid=192.168.10.11:8091:2031428229,branchType=AT,resourceId=jdbc:mysql://{IP}:${port}/data_sync,lockKey=ds_orderresourcepool:a5da7423bb35483d9ecf95a3e1291b84;ds_orderresourcepool:ebc709fa49484a4e9716c39c8ade7837;ds_orderresourcepool:bbef50a0080740f69ba9aec3850ac925;ds_orderresourcepool:e2e340b707334fa1a64e1cb1de503972;ds_orderresourcepool:7956edcf438f4867925095b012a6ccd8;ds_orderresourcepool:fdc5908dbb1e405f904fe979d21545ad;ds_orderresourcepool:96a92494668d4ed99f1af526ae2bf756 ,clientIp:192.168.10.11,vgroup:ds-resource-service-seata-service-group 2020-04-23 14:00:29.787 ERROR[ServerHandlerThread_1_500]io.seata.core.exception.AbstractExceptionHandler.exceptionHandleTemplate:120 -Catch TransactionException while do RPC, request: xid=192.168.10.11:8091:2031428229,branchType=AT,resourceId=jdbc:mysql://${IP}:${port}/data_sync,lockKey=ds_orderresourcepool:a5da7423bb35483d9ecf95a3e1291b84;ds_orderresourcepool:ebc709fa49484a4e9716c39c8ade7837;ds_orderresourcepool:bbef50a0080740f69ba9aec3850ac925;ds_orderresourcepool:e2e340b707334fa1a64e1cb1de503972;ds_orderresourcepool:7956edcf438f4867925095b012a6ccd8;ds_orderresourcepool:fdc5908dbb1e405f904fe979d21545ad;ds_orderresourcepool:96a92494668d4ed99f1af526ae2bf756 io.seata.core.exception.GlobalTransactionException: Could not register branch into global session xid = 192.168.10.11:8091:2031428229 status = Rollbacking at io.seata.server.coordinator.DefaultCore.lambda$branchRegister$0(DefaultCore.java:73) at io.seata.server.session.GlobalSession.lockAndExcute(GlobalSession.java:605) at io.seata.server.coordinator.DefaultCore.branchRegister(DefaultCore.java:70) at io.seata.server.coordinator.DefaultCoordinator.doBranchRegister(DefaultCoordinator.java:203) at io.seata.server.AbstractTCInboundHandler$4.execute(AbstractTCInboundHandler.java:172) at io.seata.server.AbstractTCInboundHandler$4.execute(AbstractTCInboundHandler.java:167) at io.seata.core.exception.AbstractExceptionHandler.exceptionHandleTemplate(AbstractExceptionHandler.java:117) at io.seata.server.AbstractTCInboundHandler.handle(AbstractTCInboundHandler.java:167) at io.seata.core.protocol.transaction.BranchRegisterRequest.handle(BranchRegisterRequest.java:136) at io.seata.server.coordinator.DefaultCoordinator.onRequest(DefaultCoordinator.java:544) at io.seata.core.rpc.DefaultServerMessageListenerImpl.onTrxMessage(DefaultServerMessageListenerImpl.java:91) at io.seata.core.rpc.netty.RpcServer.dispatch(RpcServer.java:295) at io.seata.core.rpc.netty.AbstractRpcRemoting$3.run(AbstractRpcRemoting.java:384) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-04-23 14:00:29.789 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.doGlobalRollback:418 -Successfully rollback branch xid=192.168.10.11:8091:2031428229 branchId=2031429561 2020-04-23 14:00:29.818 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.doGlobalRollback:418 -Successfully rollback branch xid=192.168.10.11:8091:2031428229 branchId=2031429560 2020-04-23 14:00:29.839 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.doGlobalRollback:418 -Successfully rollback branch xid=192.168.10.11:8091:2031428229 branchId=2031429559 2020-04-23 14:00:29.873 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.doGlobalRollback:418 -Successfully rollback branch xid=192.168.10.11:8091:2031428229 branchId=2031429558 2020-04-23 14:00:29.944 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.doGlobalRollback:418 -Successfully rollback branch xid=192.168.10.11:8091:2031428229 branchId=2031429557 2020-04-23 14:00:30.020 INFO [ServerHandlerThread_1_500]io.seata.server.coordinator.DefaultCore.doGlobalRollback:418 -Successfully rollback branch xid=192.168.10.11:8091:2031428229 branchId=2031429556

3. undo_log表的记录

INSERT INTO data_sync.undo_log (id, branch_id, xid, context, rollback_info, log_status, log_created, log_modified, ext) VALUES (93349, 2031428232, '192.168.10.11:8091:2031428229', 'serializer=jackson', 0x7B7D, 1, '2020-04-23 14:01:12', '2020-04-23 14:01:12', null);
INSERT INTO data_sync.undo_log (id, branch_id, xid, context, rollback_info, log_status, log_created, log_modified, ext) VALUES (94801, 2031429565, '192.168.10.11:8091:2031429562', 'serializer=jackson', 0x7B7D, 1, '2020-04-23 14:04:05', '2020-04-23 14:04:05', null);
INSERT INTO data_sync.undo_log (id, branch_id, xid, context, rollback_info, log_status, log_created, log_modified, ext) VALUES (96234, 2031431041, '192.168.10.11:8091:2031431038', 'serializer=jackson', 0x7B7D, 1, '2020-04-23 14:31:02', '2020-04-23 14:31:02', null);

Ⅲ. Describe what you expected to happen

我不知道为什么会报这个异常,我预期应该能够顺利保存数据

Ⅳ. How to reproduce it (as minimally and precisely as possible)

好像没发现什么规律复现.
比如同相同的数据,相同的操作,16:23的时候触发就报了这个异常,17:40的时候再次触发就成功了

Ⅴ. Anything else we need to know?

目前已经在生产环境使用seata了,希望能够获得帮助.谢谢

Ⅵ. Environment:

• JDK version : 1.8
• OS : CentOS7
• Others: seata server 1.0 , spring cloud greenwich

No repository file, test, or entry point is named. Start by reproducing the intermittent commit failure with Spring Data JPA in Seata AT/file mode, then compare the application and Seata Server logs around the JDBC commit. Done means isolating the trigger and documenting or testing a correction.