proposal: have 2nd, 3rd attempt for non-starter = timeout runs

Open
#899 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
backend

Research direction

Review issue #897 and pScheduler’s handling of newly received tasks, coordination timeouts, and non-starter runs. The work would first need an agreed retry policy, including postponed second or third attempts and any load checks; it is done when that behavior is specified and implemented with coverage for the proposed conditions.

Written by the indexing model from the issue text.

Description

enhancement

Here are some thoughts on my side as suggestion to improve the management of 'non-starter' runs on systems under heavy load.
Since I know from you pScheduler is aware of the reasons for each 'non-starter', which I presume includes time-out outcomes, I was thinking, that at least in the cases where both conditions apply:

  • The pscheduler task had just been received.
  • Pscheduler times out on runs coordination attempt

IMHO It would then be more reasonable to have pscheduler make a second attempt for these runs.
In special cases even a third attempt would be acceptable, if the task is of type background or normal.
Of course these attempts should be postponed in time and perhaps (for pS 4.6 ) pscheduler can check even the system’s load ( ex. load average for last 5min) to allow itself to reiterate the “non-starter” runs, which had timed-out.

This suggestion comes in relation to https://github.com/perfsonar/pscheduler/issues/897

Dominant language
Python
Stars
60
Forks
38
Avg merge
1m
Merged PRs (30d)
1

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from perfsonar/pscheduler

All issues in perfsonar/pscheduler

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.