argoproj / argoproj/argo-workflows

workflow step name in keyFormat for archiveLogs

Open
#3,417 3 comments 28 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/archive-logs area/artifacts
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

Summary

I would like to use {{step.name}} in keyFormat definition when using archiveLogs: true.

Motivation

Currently the only way to separate the logs from a single workflow and preventing Argo from overwrite the logs from the preceding workflow steps I have to use {{pod.name}}. pod.name does not provide information about the step the log file was created by. This requires searching through all log files in each pod name folder to find the right output.

Proposal

When setting keyFormat: {{workflow.label.data-id}}/{{workflow.name}}-{{workflow.creationTimestamp}}/{{step.name}}.log Argo should create a single log file named after the steps name in that specified folder.
Example:
"A123/preprocessing-2020-07-08 12:11:53 +0000 UTC/merge-files.log"
This requires that:

  1. Argo detects "keyFormat" ends with ".log" so it does not create a subfolder with file "main.log"
  2. Argo is able to translate the variable "step.name".

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from the archiveLogs and keyFormat handling described in the issue, including template variable translation and the requested .log behavior. Trace how archived workflow logs currently derive pod folders and main.log, then verify that step.name produces one step-named log file in the requested folder.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.