argoproj / argoproj/argo-workflows

If a running workflow is deleted, workflow archive is not created

Open
#13,658 1 comment 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/workflow-archive
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

Pre-requisites
  • I have double-checked my configuration
  • I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
  • I have searched existing issues and could not find a match for this bug
  • I'd like to contribute the fix myself (see contributing guide)
What happened? What did you expect to happen?

We manage workflow resources directly in GitOps.
In an environment where ArchiveWorkflow is enabled, if there is a workflow in the running state and the workflow is deleted by GitOps, the archive is not created even though ArchiveWorkflow is enabled.

This is due to the workflow archive mechanism in Argo Workflows.
Workflow controller triggers the workflow archive when the workflow phase is marked as Succeeded or Failed.
https://github.com/argoproj/argo-workflows/blob/554fb0c10eb38f23b170b38870cbc48e522dfd2f/workflow/controller/operator.go#L2411

However, if a running workflow is deleted, the workflow archive is not created because the workflow phase is deleted without being marked as any phase.
Thus, the target workflow will be forever untraceable.
This problem could potentially occur in non-GitOps environments as well.

I think the workflow phase should be marked as one of the end phases before deleting the workflow. (i.e. the workflow archive is created).
However, I could not think of a good way to block workflow deletion until the workflow phase is changed.

Version(s)

v3.5.11

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
# Delete workflow during running status
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-
spec:
  entrypoint: hello-world
  templates:
  - name: hello-world
    container:
      image: busybox
      command: [sleep]
      args: ["10000"]
Logs from the workflow controller
None
Logs from in your workflow's wait container
None

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in workflow/controller/operator.go around the archive trigger linked in the issue, then reproduce with the provided busybox workflow by deleting it while it is running. Trace how deletion affects the workflow phase and archive handling. Done means a deleted running workflow produces an archive when ArchiveWorkflow is enabled, with coverage for the reproduction case.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.