citusdata / citusdata/pg_cron

Tasks beyond max_running_tasks do not get a chance to execute

Open
#386 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
3.9k
Forks
260
Avg merge
5d 5h
Merged PRs (30d)
1

Description

Description:

In the following code snippet:
```sql
foreach(taskCell, taskList)
{
CronTask *task = (CronTask *) lfirst(taskCell);
PostgresPollingStatusType pollingStatus = task->pollingStatus;
struct pollfd *pollFileDescriptor = &pollFDs[activeTaskCount];

if (activeTaskCount >= max_running_tasks)
{
/* already polling the maximum number of tasks */
break;
}
}
```
It seems that any task beyond the max_running_tasks limit never gets a chance to execute, because the loop breaks as soon as the active task count reaches the limit. This raises the concern that tasks later in the list might be completely ignored, especially if activeTaskCount is never decremented or reset elsewhere for polling new tasks.

Could you clarify:
• Is there any logic that allows the skipped tasks to be retried in a subsequent cycle?
• If not, should the code be changed to process all tasks, even if only a limited number can be active at a time?

This might lead to starvation for tasks that are always queued behind the currently running ones.

Thanks!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the polling loop shown in the issue and trace how activeTaskCount changes across polling cycles. Verify whether tasks after the max_running_tasks limit are revisited; the work is done when the retry behavior is established and any starvation is resolved or clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.