RejectionException while shutting down Ebean and NotifyOfCommit should be scheduled
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 267
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 5
Description
Application server running in Docker with active docker health check sometimes causes application server to restart (memory leaks in JVM). The restart is a regular application server shutdown and not a pure kill -9. In that case Ebean will be shut down and often throw a RejectionException during some commits that might still be executed on the server.
I am wondering if that is a race condition that might need to be fixed. I could imagine the code flow is like
- commit()
- db.shutdown()
- notifyCommit(), which calls an already terminated scheduler causing the exception
Maybe the exception should be handled in Ebean, if it is expected to be thrown during shutdown.
ERROR i.e.s.transaction.TransactionManager
NotifyOfCommit failed. L2 Cache potentially not notified.
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@1a6dea8d[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@370f053c[Wrapped task = io.ebeaninternal.server.executor.DefaultBackgroundExecutor$$Lambda$708/0x0000000801193800@62db3d40]] rejected from io.ebeaninternal.server.executor.DaemonScheduleThreadPool@7b14312[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8564]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2057)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:827)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.submit(ScheduledThreadPoolExecutor.java:715)
at io.ebeaninternal.server.executor.DefaultBackgroundExecutor.submit(DefaultBackgroundExecutor.java:75)
at io.ebeaninternal.server.executor.DefaultBackgroundExecutor.execute(DefaultBackgroundExecutor.java:80)
at io.ebeaninternal.server.transaction.TransactionManager.notifyOfCommit(TransactionManager.java:452)
at io.ebeaninternal.server.transaction.JdbcTransaction.notifyCommit(JdbcTransaction.java:934)
at io.ebeaninternal.server.transaction.JdbcTransaction.postCommit(JdbcTransaction.java:1005)
at io.ebeaninternal.server.transaction.JdbcTransaction.flushCommitAndNotify(JdbcTransaction.java:999)
at io.ebeaninternal.server.transaction.JdbcTransaction.commit(JdbcTransaction.java:1058)
at io.ebeaninternal.api.ScopeTrans.commitTransaction(ScopeTrans.java:140)
at io.ebeaninternal.api.ScopedTransaction.commit(ScopedTransaction.java:110)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading TransactionManager.notifyOfCommit and the shutdown behavior of DefaultBackgroundExecutor. Trace the commit path through JdbcTransaction.postCommit, flushCommitAndNotify, and commit to confirm whether notification can reach a terminated scheduler. Done means establishing and testing behavior for commits during shutdown, with no unexpected RejectedExecutionException or lost cache notification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100