Make work order-specific queues blocking (Run Concurrency)
Open
Nobody has claimed this yet.
needs detail
- Dominant language
- Elixir
- Stars
- 296
- Forks
- 86
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 50
Description
Make work order-specific queues blocking (Enforce order/sequence);
Via the workflow settings panel, we want to enable users to the able to:
- Specify that for a given workflow, work order 3 should not be processed until work order 2 reaches
state = success || ignored - Allow users to specify that for a given workflow, work order 7 should not be processed until work order 4 reaches
state = success || ignoredif both meet the same key criterion. (This feature is required by Kafka trigger as well to enforce sequence in the how messages are processed by the consumer group)
Select from runs
where state = available
and no other workorders for this workflow with dataclip.body.arbitrary_key is yet
to have state = [success|skipped]
- Allow users to specify concurrency per-arbitrary-dataclip-key-per-workflow.
So if we have 10 runs enqueued (3 with the same arbitrary-data clip-value: dataclip.body.xyx = “myguy!”)
then we’d do 8-at-a-time, but keep the 2 other dataclip.body.xyx = “myguy!” containing runs BLOCKED
until the first was done.
- Allow users to mark work orders in the ignored state
update work order set state = ‘ignored’ where wo.id = this_wo_id
- Allow users to clear (set to ignore, never process) pending runs/workorders in their queue
update runs set state = ‘skipped’ where runs.state = ‘available’
update work order state if the skipped run is only run for that workorder
- Maybe make it easier to find work orders to retry that include skipped runs?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the workflow settings panel, the run/work-order queue behavior, and the Kafka trigger or consumer-group entry point named in the issue. The issue names no files or tests; done would require agreeing on and implementing the six listed ordering, concurrency, ignored-state, clearing, and retry behaviors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100