conductor-oss / conductor-oss/conductor
Http Task not Picked soon
- Dominant language
- Java
- Stars
- 32.2k
- Forks
- 1k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 37
Description
**Describe the bug**
We are executing 100 Workflows per second in 4 Containers pods with 4 core in each pod. We observe that Http Task is not picked immediately. Updated activeWorkerLastPollTimeout to 2 sec where we saw improvements but not great improvement. Wonder is there any other setting we are missing.

In the above example Task2 is picked after 9 seconds and hence the join took more time.
**Details**
Conductor version: v3.21.5
Persistence implementation: Cassandra, Postgres, MySQL, Dynomite etc Postgres
Queue implementation: Postgres, MySQL, Dynoqueues etc Postgres
Lock: Redis or Zookeeper?
Workflow definition:
Task definition:
Event handler definition:
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
Http Task pulled immediately and better metrics on pending Http Task Qeueu
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Additional context**
We are comparing this with AWS Step Functions and this issue is holding us to move forward with Conductor
Contributor guide
Assessment
This issue has not been assessed yet.