get2knowio / get2knowio/maverick

Add trace ID correlation to workflow events

Open
#18 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
4
Forks
0
Avg merge
17h 37m
Merged PRs (30d)
7

Description

## Summary

Add a `trace_id` field to the `ProgressEvent` base to enable end-to-end correlation of all events across a workflow execution, improving post-mortem debugging and observability.

## Motivation

Maverick's event system emits rich, typed events during workflow execution (`StepStarted`, `StepCompleted`, `AgentStreamChunk`, `LoopIterationStarted`, etc.). However, there is no single correlation identifier that ties all events from one workflow run together. This makes it difficult to:

- Filter log output to a specific run when multiple runs overlap
- Correlate events in the session journal with external system interactions (GitHub API calls, git operations)
- Build post-mortem timelines from structured log data
- Aggregate metrics across all events in a single run

### Inspiration from Oracle's WayFlow

Oracle's [WayFlow](https://oracle.github.io/wayflow/25.4.2/) has explicit tracing support ("Enable Tracing" how-to guide). While Maverick's event system is already richer than WayFlow's tracing (step-level events, streaming chunks, loop iteration tracking), the gap is **correlation** — linking all events from a single execution with a shared identifier.

The existing `--session-log` feature partially addresses this by writing events to a file, but the events themselves don't carry a trace ID, making programmatic filtering and aggregation harder.

## Proposed Change

### Add `trace_id` to event base

Every `ProgressEvent` should carry a `trace_id: str` field. This is a UUID generated once at the start of `WorkflowFileExecutor.execute()` and propagated to all events emitted during that execution.

```python
@dataclass(frozen=True)
class WorkflowStarted:
workflow_name: str
inputs: dict[str, Any]
timestamp: str
trace_id: str # NEW: shared across all events in this run
```

### Propagation

1. `WorkflowFileExecutor.execute()` generates a `trace_id = str(uuid4())` at the start
2. Passes it to all event constructors via the `event_callback` wrapper
3. Subworkflow steps inherit the parent's `trace_id` (so nested workflows share the same trace)
4. The `trace_id` is included in `to_dict()` serialization

### Integration with session log

The session journal (`--session-log`) should include the `trace_id` in its header/metadata, making it trivial to correlate the log file with events:

```json
{
"trace_id": "a1b2c3d4-...",
"workflow": "feature",
"started_at": "2026-02-05T10:30:00Z",
"events": [...]
}
```

### Integration with structlog

Add `trace_id` as a structlog context variable so all log lines during a workflow execution are tagged:

```python
logger = get_logger(__name__)
structlog.contextvars.bind_contextvars(trace_id=trace_id)
```

This means even log lines from deep utility code (git operations, GitHub API calls) will carry the trace ID without explicit threading.

## Scope

- Add `trace_id: str` field to all `ProgressEvent` dataclasses
- Generate trace ID in `WorkflowFileExecutor`
- Propagate through event callbacks and subworkflow execution
- Include in `to_dict()` serialization
- Bind to structlog context variables
- Include in session log metadata

## Acceptance Criteria

- [ ] `trace_id` field on all `ProgressEvent` types
- [ ] UUID generated at workflow execution start
- [ ] Propagated to all events including nested subworkflows
- [ ] Included in `to_dict()` serialization
- [ ] Bound to structlog context for correlated logging
- [ ] Session log includes trace ID in metadata
- [ ] Tests verify trace ID consistency across event stream
- [ ] No breaking changes to existing event consumers (field is additive)

## References

- Oracle WayFlow tracing: https://oracle.github.io/wayflow/25.4.2/core/howtoguides/index.html
- Current event system: `src/maverick/dsl/events.py`
- Current session log: `--session-log` CLI option
- structlog contextvars: https://www.structlog.org/en/stable/contextvars.html

Contributor guide

Open the contributing guide

Research direction

Start with src/maverick/dsl/events.py and trace WorkflowFileExecutor.execute() to understand how ProgressEvent instances and event callbacks are created. Then locate the --session-log handling and structlog setup. Done means every event, including nested workflow events, shares one serialized trace ID, and session logs and log context expose it without breaking existing consumers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.