pulp / pulp/pulp_workflow

Add a manual purge action for completed Workflows

Open
#5 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
1
Forks
3
Avg merge
4d 8h
Merged PRs (30d)
4

Description

Summary

Add a manual API for purging completed Workflow records, mirroring the existing pulpcore POST /pulp/api/v3/tasks/purge/ action.

Motivation

Workflows accumulate the same way tasks do — once completed, the rows stay around for audit/history but eventually become noise. Pulpcore already provides this for tasks via the tasks_purge action; workflows should have the equivalent so operators have a way to keep the workflow table tidy.

Explicitly manual, not automatic: like pulpcore's task purge, the action should only run when an operator (or a scheduled job they configure themselves) invokes it. No background sweeping should be added.

How pulpcore does it (for reference)

  • POST /pulp/api/v3/tasks/purge/ is an @action(detail=False, methods=["post"]) on TaskViewSet (pulpcore/app/viewsets/task.py).
  • The action accepts a PurgeSerializer body with two fields (pulpcore/app/serializers/purge.py):
    • finished_before — datetime; defaults to 30 days ago.
    • states — set of allowed final states; defaults to ["completed"], restricted to TASK_FINAL_STATES.
  • The action dispatches pulpcore.app.tasks.purge.purge and returns an OperationPostponedResponse (202).
  • The purge task is itself a regular pulp task: it filters in chunks of 1000, scopes by current domain and core.delete_task permission, and emits ProgressReports for totals, per-type counts, and errors.
  • A tasks entry is registered in INITIAL_GUARDED_SCHEDULES (pulpcore/app/util.py) so a default schedule exists, but the user is in control of whether it actually runs.

Proposed design for pulp_workflow

Add a purge action on WorkflowViewSet:

  • POST /pulp/api/v3/workflows/purge/ (action, detail=False, methods=["post"]).
  • Request body — WorkflowPurgeSerializer:
    • finished_beforeDateTimeField, default now() - 30d. Only workflows whose finished_at is strictly less than this are eligible.
    • statesMultipleChoiceField over the workflow's terminal states (COMPLETED, FAILED, CANCELED); default ["completed"].
  • Dispatch a new task pulp_workflow.app.tasks.purge_workflows and return OperationPostponedResponse (202). Do not run synchronously.
  • Permission model:
    • Add a delete_workflow Django permission to Workflow.Meta.permissions (or rely on the default delete permission once added) and gate the action on it via DEFAULT_ACCESS_POLICY, mirroring pulpcore's pattern of filtering by delete_task inside the purge task.
    • Scope by current domain (pulp_domain=get_domain()) when the task was dispatched by a user (i.e. not from a schedule).
  • Inside purge_workflows:
    • Build a queryset of Workflow.objects.filter(finished_at__lt=finished_before, state__in=states).
    • Delete in chunks of 1000 to keep memory bounded (same DELETE_LIMIT pattern as pulpcore).
    • Emit ProgressReports with codes such as purge.workflows.total, purge.workflows.key.<model>, purge.workflows.error for parity with the task purge.
  • Cascade behavior: WorkflowTask, WorkflowTaskArg, and WorkflowTaskKwarg already cascade off Workflow via on_delete=CASCADE, so deleting the Workflow cleans those up automatically. The associated TaskGroup (once #3 lands) and the child Task rows are not owned by the workflow and should be left alone — they are governed by tasks_purge. Document this in the action's help text so users understand they need to run both.

Explicitly out of scope

  • No automatic/scheduled purging by default. Do not add a TaskSchedule entry that runs on its own. Operators who want recurring purges can create their own TaskSchedule pointing at pulp_workflow.app.tasks.purge_workflows.
  • No purge of running or waiting workflows. The states field must reject non-terminal states (validate in the serializer the same way pulpcore's PurgeSerializer restricts to TASK_FINAL_STATES).
  • No deletion of the Task rows the workflow dispatched. Those are pulpcore's responsibility.

Acceptance criteria

  • POST /pulp/api/v3/workflows/purge/ exists, accepts finished_before and states, returns 202 with a task reference.
  • Dispatched task removes only workflows whose finished_at < finished_before and whose state is in the (validated terminal) states list.
  • Workflows in non-terminal states are never deleted.
  • Cascading delete of WorkflowTask / WorkflowTaskArg / WorkflowTaskKwarg works.
  • Default finished_before is "30 days ago" and default states is ["completed"].
  • Domain scoping matches pulpcore's task purge (current-domain only when dispatched by a user).
  • Permission check prevents users without delete_workflow from purging.
  • Functional test covers: defaults spare recent workflows; explicit finished_before in the future deletes everything terminal; non-terminal workflows are spared regardless.

Related

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing WorkflowViewSet, WorkflowPurgeSerializer, and the proposed purge_workflows task with pulpcore's task.py, purge.py, and serializers/purge.py reference implementations. Done means the purge endpoint returns 202, validates terminal states and permissions, applies the stated domain and date filters, cascades workflow-owned rows, and has the described functional coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
api, authorization, backend, database, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.