Identifiy and fix broken cron workflows
Nobody has claimed this yet.
- Dominant language
- Elixir
- Stars
- 296
- Forks
- 86
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 50
Description
We have a problem where empty cron workflows recurse forever - see https://github.com/OpenFn/kit/issues/1367
I have put a bandaid on the problem for new workflows. But existing ones may still give us problems.
This issue is two-fold:
- identify any existing workflows on prod and staging which may trigger this behaviour
- roll out a fix
Identification
We need to work out which workflows are affected.
These things must be true:
- The workflow must be on a cron trigger
- The workflow must have empty bodies (empty is not
nil, it could just mean a comment) - The workflow must have multiple leaf nodes. Probably not enough to just have true/false branches - it's gotta have true/true edges
I don't know which of these we can use to build a query - some of them are quite hard heuristics
What might help is: the worker will detect runs with multiple leaf nodes. If a run does NOT have multiple leaves, it'll include final_dataclip_id. So maybe: identify cron runs which did not return final_dataclip_id in the final payload - those are the ones we need to fix.
For the record, this affects at least one workflow on prod: https://app.openfn.org/projects/57862287-23e6-4650-8d79-e1dd88b24b1c/w/0f1d0b6e-8426-4023-959d-a89ba824f3c0
Fix
Consider the following:
- Edit the last dataclip - the source for the next cron - to be an empty object
{} - Delete or edit prior dataclips, which could be taking up a lot of storage and providing no value
- Disabling the cron trigger (bear in mind that if a user switches it back on, they'll just provoke the problem again)
When deciding how aggressively to fix this, we need to consider if we're looking at demo code or a real project.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the worker behavior described in OpenFn/kit#1367 and inspect cron runs whose final payload lacks final_dataclip_id. Identify affected workflows on production and staging using the listed trigger, empty-body, and multiple-leaf criteria; done means affected workflows are found and their dataclips or cron triggers are safely fixed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100