flowable / flowable/flowable-engine
Endless job processing loop when using JMS/ActiveMQ
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 7h 8m
- Merged PRs (30d)
- 2
Description
**Describe the bug**
When using JMS/ActiveMQ as per https://flowable.com/open-source/docs/bpmn/ch18-Advanced/#message-queue-based-async-executor, the Jobs can get into an endless lock fail / retry loop.
This initially surfaced as a lock failure described in this forum thread https://forum.flowable.org/t/update-on-act-ru-execution/4949. After deeper investigation, the traceback included
```
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30)
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56)
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51)
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.lockJob(ExecuteAsyncRunnable.java:190)
at org.flowable.job.service.impl.asyncexecutor.ExecuteAsyncRunnable.run(ExecuteAsyncRunnable.java:113)
at org.flowable.spring.executor.jms.JobMessageListener.onMessage(JobMessageListener.java:40)
```
And with a little debug logging we are able to track this down to a single job ID that retries 100s of times per second.
**Expected behavior**
The job should (eventually) fail and be put into the `ACT_RU_DEADLETTER_JOB` table.
**Code**
We use the example https://flowable.com/open-source/docs/bpmn/ch18-Advanced/#message-queue-based-async-executor to configure our Async JMS integration.
**Additional context**
Flowable Version : 6.5.0
Flowable in a Spring container, but NOT Spring Boot.
ActiveMQ
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with ExecuteAsyncRunnable.lockJob and JobMessageListener.onMessage, using the message-queue-based async executor configuration linked in the issue. Reproduce the lock-failure retry loop with JMS/ActiveMQ and inspect how failed jobs are retried. Done means a persistently failing job eventually appears in ACT_RU_DEADLETTER_JOB instead of retrying indefinitely.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100