citusdata / citusdata/pg_cron

PG_Cron Container Scope of depended job

Open
#260 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
C
Stars
3.9k
Forks
260
Avg merge
5d 5h
Merged PRs (30d)
1

Description

Coming from MS SQL Server world I used to design packages though integration services using C# or by using the IDE (DataTools), while I was able to perform a few tasks in parallel and then execute some other tasks, in serial or in parallel, but after the first parallel stage of tasks like the following:
![image](https://user-images.githubusercontent.com/6669704/229102089-cdff0874-3aa3-414f-84b3-fabc7b8a9529.png)
*image from my custom product software

Working last year with PostgreSQL and this nice scheduler PG_Cron, I see that there is possibility to archive a similar thing out of the box while this should handled by the user manipulating some data in some special tables keeping the status of jobs with some iteration id flags while performing the following:
![image](https://user-images.githubusercontent.com/6669704/229102657-5c3faeff-8a1b-44c5-9271-a3b468690696.png)
*image from https://stackoverflow.com/questions/68598866/how-to-run-postgres-pg-cron-job-after-another-job

What I suggest is a new property called scope which can be part (column) of the table cron.job adn it will specify the scope of each job. This way we may have multiple jobs for the same scope. Then we can have another job which can be triggered when all jobs have finished from scopex. The configuration of the schedule for depended jobs can be something like "scopex" instead of "* * * * *".

In summary we may have multiple scopes containing multiple jobs and when one scope, scopeA, is finish then the other group of jobs in scopeB will trigger for execution and so one...

This will be great enhancement

Contributor guide

No contributing guide indexed for this repository

Research direction

The proposal names the cron.job table and scope-based scheduling; start by reviewing how pg_cron stores jobs and determines when scheduled jobs run. Done means supporting multiple jobs per scope and triggering a dependent scope after all jobs in the preceding scope finish, with the behavior covered by appropriate validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.