conductor-oss / conductor-oss/conductor

Http Task not Picked soon

Open
#376 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
32.2k
Forks
1k
Avg merge
2d 3h
Merged PRs (30d)
37

Description

**Describe the bug**
We are executing 100 Workflows per second in 4 Containers pods with 4 core in each pod. We observe that Http Task is not picked immediately. Updated activeWorkerLastPollTimeout to 2 sec where we saw improvements but not great improvement. Wonder is there any other setting we are missing.

![Image](https://github.com/user-attachments/assets/24a57884-44ba-48f2-8ad3-0bfde77f0b8a)

In the above example Task2 is picked after 9 seconds and hence the join took more time.

**Details**
Conductor version: v3.21.5
Persistence implementation: Cassandra, Postgres, MySQL, Dynomite etc Postgres
Queue implementation: Postgres, MySQL, Dynoqueues etc Postgres
Lock: Redis or Zookeeper?
Workflow definition:
Task definition:
Event handler definition:

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
Http Task pulled immediately and better metrics on pending Http Task Qeueu

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Additional context**
We are comparing this with AWS Step Functions and this issue is holding us to move forward with Conductor

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.