Configure automatic retry for failed runs

Goal:

1. Make developers only worry about truly broken work orders

KPI: Consultant NPS

Problem/Observation

There are times when a work order fails for some external reasons (which can also be temporary) and project users who have configured their projects to send failure alerts receive an email.

Users can manually retry their runs via the history page, but they can't set it to work automatically.

The factors causing the failed run attempt might include app crashes, rate-limiting errors, or validation errors and would not require users to rewrite/refactor their code for a fix.

In this case, we want users to be able to configure work orders of a workflow in their project to automatically retry by specifying the number of retries.

What to be done:

Via the workflow configuration, I would love to be able to

1. Configure my workflow to allow automatic retries by:
   a. Toggling it on
   b. Setting the number of times it should retry
   c. Setting the retry delay period.

When retry is enabled for a workflow:

1. Automatic retries should not apply for manual runs i.e. if I triggered a run myself
2. I should receive an email:
   a. when the job fails (the regular failure alerts) and retries have all failed
   b. When the retry is successful (It would be great to see which attempt succeeded)
   c. If the final retry fails

No files, tests, or entry points are named. Trace the workflow configuration, run history/manual-run behavior, and failure-alert email paths first; done means configurable retry enablement, count, and delay, with manual runs excluded and the specified success and final-failure notifications sent.