argoproj / argoproj/argo-workflows

Invalid node IDs in workflow.status

Open
#10,107 24 comments 6 reactions 0 assignees View on GitHub
P1 type/regression
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

### Pre-requisites

- [X] I have double-checked my configuration
- [X] I can confirm the issues exists when I tested with `:latest`
- [ ] I'd like to contribute the fix myself (see [contributing guide](https://github.com/argoproj/argo-workflows/blob/master/docs/CONTRIBUTING.md))

### What happened/what you expected to happen?

Since #8748 pod names use the V2 naming which contains the template name in the pod name. However the implementation did not update the Workflow.Status.Nodes map to contain the correct pod name anymore. There's a disconnect between NodeIDs and pod names which wasn't the case before. This makes it impossible to look at a argo workflow status, take the nodeID and use it to know which pod it belongs to.

This is a follow-up of #9906. I initially thought this would be the same case but apparently is not.

The below workflow triggers the following workflow:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
annotations:
workflows.argoproj.io/pod-name-format: v2
creationTimestamp: "2022-11-25T11:33:41Z"
generateName: nodename-
generation: 3
labels:
workflows.argoproj.io/phase: Running
name: nodename-bvd45
namespace: argo
resourceVersion: "15649"
uid: ea233eef-210d-4394-a238-ef847b104458
spec:
activeDeadlineSeconds: 300
arguments: {}
entrypoint: render
podSpecPatch: |
terminationGracePeriodSeconds: 3
templates:
- inputs: {}
metadata: {}
name: render
outputs: {}
steps:
- - arguments:
parameters:
- name: frames
value: '{{item.frames}}'
name: run-blender
template: blender
withItems:
- frames: 1
- container:
args:
- /argosay echo 0/100 $ARGO_PROGRESS_FILE && /argosay sleep 10s && /argosay
echo 50/100 $ARGO_PROGRESS_FILE && /argosay sleep 10s
command:
- /bin/sh
- -c
image: argoproj/argosay:v2
name: ""
resources: {}
inputs:
parameters:
- name: frames
metadata: {}
name: blender
outputs: {}
retryStrategy:
limit: 2
retryPolicy: Always
status:
artifactGCStatus:
notSpecified: true
artifactRepositoryRef:
artifactRepository:
archiveLogs: true
s3:
accessKeySecret:
key: accesskey
name: my-minio-cred
bucket: my-bucket
endpoint: minio:9000
insecure: true
secretKeySecret:
key: secretkey
name: my-minio-cred
configMap: artifact-repositories
key: default-v1
namespace: argo
conditions:
- status: "False"
type: PodRunning
finishedAt: null
nodes:
nodename-bvd45:
children:
- nodename-bvd45-701773242
displayName: nodename-bvd45
finishedAt: null
id: nodename-bvd45
name: nodename-bvd45
phase: Running
progress: 0/1
startedAt: "2022-11-25T11:33:41Z"
templateName: render
templateScope: local/nodename-bvd45
type: Steps
nodename-bvd45-701773242:
boundaryID: nodename-bvd45
children:
- nodename-bvd45-3728066428
displayName: '[0]'
finishedAt: null
id: nodename-bvd45-701773242
name: nodename-bvd45[0]
phase: Running
progress: 0/1
startedAt: "2022-11-25T11:33:41Z"
templateScope: local/nodename-bvd45
type: StepGroup
nodename-bvd45-3728066428:
boundaryID: nodename-bvd45
children:
- nodename-bvd45-3928099255
displayName: run-blender(0:frames:1)
finishedAt: null
id: nodename-bvd45-3728066428
inputs:
parameters:
- name: frames
value: "1"
name: nodename-bvd45[0].run-blender(0:frames:1)
phase: Running
progress: 0/1
startedAt: "2022-11-25T11:33:41Z"
templateName: blender
templateScope: local/nodename-bvd45
type: Retry
nodename-bvd45-3928099255:
boundaryID: nodename-bvd45
displayName: run-blender(0:frames:1)(0)
finishedAt: null
hostNodeName: k3d-argowf-server-0
id: nodename-bvd45-3928099255
inputs:
parameters:
- name: frames
value: "1"
message: PodInitializing
name: nodename-bvd45[0].run-blender(0:frames:1)(0)
phase: Pending
progress: 0/1
startedAt: "2022-11-25T11:33:41Z"
templateName: blender
templateScope: local/nodename-bvd45
type: Pod
phase: Running
progress: 0/1
startedAt: "2022-11-25T11:33:41Z"
```

The pod to run is named `nodename-bvd45-blender-3928099255` but the `NodeID` in `workflow.status.nodes` is just `nodename-bvd45-3928099255`.

### Version

latest

### Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

```YAML
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: nodename-
spec:
arguments: {}
entrypoint: render
templates:
- inputs: {}
metadata: {}
name: render
steps:
- - arguments:
parameters:
- name: frames
value: '{{item.frames}}'
name: run-blender
template: blender
withItems:
- frames: 1
- container:
image: argoproj/argosay:v2
command: ["/bin/sh", "-c"]
args:
- /argosay echo 0/100 $ARGO_PROGRESS_FILE && /argosay sleep 10s && /argosay echo 50/100 $ARGO_PROGRESS_FILE && /argosay sleep 10s
name: ""
inputs:
parameters:
- name: frames
name: blender
retryStrategy:
limit: 2
retryPolicy: Always
```

### Logs from the workflow controller

```text
irrelevant
```

### Logs from in your workflow's wait container

```text
irrelevant
```

Contributor guide

Open the contributing guide

Research direction

Use the supplied workflow YAML to reproduce the V2 pod-name case, then inspect the workflow controller path that updates workflow.status.nodes and the pod-name generation behavior described in the issue. Done means the node ID in status.nodes corresponds to the generated pod name, including the template name, without breaking the provided workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
backend, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.