argoproj / argoproj/argo-workflows

Access node logs as an artifact

Open
#7,375 1 comment 4 reactions 0 assignees View on GitHub
area/archive-logs area/artifacts
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

# Summary

Accessing a nodes output via `steps.pipeline.outputs.result` works great when a node's stdout is simple, but it doesn't work as well when the stdout is volumous or is complex (for example, a traceback with lots of odd characters). It also does not include contents of stderr. It would be nice if it were also possible to access the stdout/stderr outputs of a node as an artifact (e.g. under something like `steps.pipeline.outputs.log_artifact`), which can then be shared with follow-up tasks to do processing on that, post it somewhere, or send out specialized notifications.

To note, this is in principle possible already with the built-in artifacting system. But this requires first writing the logs to the external storage (e.g. S3), and then reading it back in in the next step. It would be nice if this was possible without leaving the k8s/argo domain.

# Use Cases

When would you use this?
* Automatic processing of error logs
* Posting logs into a slack channel, or similar
* Automated notifying of key individuals/teams depending on the contents of logs

---

**Message from the maintainers**:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing how steps.pipeline.outputs.result and the built-in artifacting system handle node output, including stdout and stderr. No implementation files or tests are named; done means defining and implementing a way for follow-up tasks to access node logs as an artifact without round-tripping through external storage.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
backend, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.