microsoft / microsoft/conductor

feat(runtime): allow event log output directory to be configured in workflow.yaml

Open
#475 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
448
Forks
65
Avg merge
1d 18h
Merged PRs (30d)
39

Description

Summary

Event logs are always written to $TMPDIR/conductor/ (typically /tmp/conductor/) with no way to configure the output directory.

Problem

・Event logs in tmp are lost on reboot or tmpwatch cleanup
・Users must manually copy event logs to a persistent directory after each run
・When a workflow crashes without a workflow_failed event, the only diagnostic is the event log -- which may already be gone
・The event log subscriber is always-on by design (correct -- it's a flight recorder), but the output location should be user-configurable
・Configurable log output directories are standard in self-hosted orchestration tools: Airflow(base_log_folder), Argo workflows(artifactRepository), Temporal, Prefect, Docker(log-driver), Kubernetes(--log-dir)

Proposal

Add an event_log_dir field to the runtime: block in workflow YAML.

workflow:
  runtime:
    event_log_dir: ./logs

When omitted, behavior is unchanged($TMPDIR/conductor/).
Relative paths are resolved against the workflow file's directory.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at runtime configuration handling for the workflow.yaml runtime block and the always-on event log subscriber. Trace how the workflow file directory is available for relative paths; done means event_log_dir is accepted, defaults to $TMPDIR/conductor/, and resolves relative paths against the workflow file directory.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, observability-sre
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.