whiteducksoftware / whiteducksoftware/flock

[1.0] Bound incomplete AND, batch and join input collections

Open
#441 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
120
Forks
14
Avg merge
19h 32m
Merged PRs (30d)
8

Description

Work can consume memory before a run exists. Limiting only scheduled runs does not bound the ArtifactCollector and other incomplete input groups.

Scope

  • Bound held artifact references per subscription/group, including ordinary AND gates, batches and joins.
  • Count references rather than only group objects; define explicit admission outcomes when collection capacity is exhausted.
  • Preserve visibility, matching semantics and applicable expiration behavior; do not force all AND gates into correlation partitions.

Acceptance criteria

  • Repeated A-only publication to consumes(A, B) retains at most the configured references and visibly throttles or rejects further admission.
  • Batch/join cases also stay bounded; rejection does not falsely confirm a stored artifact as admitted to processing.
  • Completion, expiry and explicit disposal release the relevant capacity without dropping accepted inputs silently.

Boundaries

Persisting enough receipts to show interrupted intake is a later dependent 1.0 task; restoring full collections after restart is post-1.0.

References

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read src/flock/orchestrator/artifact_collector.py, src/flock/orchestrator/scheduler.py, and src/flock/core/subscription.py, along with dependency #440. Trace how incomplete artifact groups are admitted and released for AND, batch, and join cases. Done means configured reference bounds, visible throttling or rejection, and correct release on completion, expiry, or disposal without falsely admitting rejected inputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.