.NET Workflows: Feature Request: Pipeline Behavior Extension Points for Cross-Cutting Concerns

# Feature Request: Pipeline Behavior Extension Points for Workflow Framework

## Problem Statement

The `Microsoft.Agents.AI.Workflows` framework currently lacks extensibility hooks for cross-cutting concerns at the workflow and executor execution levels. While an event system exists (`ExecutorInvokedEvent`, `SuperStepStartedEvent`, etc.), events are fire-and-forget and **cannot**:

- Intercept or modify execution flow
- Short-circuit execution based on validation
- Measure execution time and inject telemetry
- Transform inputs/outputs
- Implement retry or caching logic
- Add authentication/authorization checks

This limitation forces developers to either:
1. Fork and modify framework internals
2. Implement workarounds with reflection
3. Duplicate cross-cutting logic across every executor

## Proposed Solution

Add **pipeline behavior extension points** that allow developers to inject custom logic before and after:
- **Workflow start/end** - When workflow execution begins and completes
- **Executor step start/end** - When individual executors process messages

This enables a middleware-style pattern (similar to ASP.NET Core or MediatR's `IPipelineBehavior`) where multiple behaviors can be chained together.

## API Design

### 1. Workflow-Level Behaviors

```csharp
public interface IWorkflowBehavior
{
ValueTask HandleAsync(
WorkflowBehaviorContext context,
WorkflowBehaviorDelegate next,
CancellationToken cancellationToken);
}

public class WorkflowBehaviorContext
{
public string WorkflowName { get; init; }
public string? WorkflowDescription { get; init; }
public string RunId { get; init; }
public string StartExecutorId { get; init; }
public WorkflowStage Stage { get; init; } // Starting or Ending
public IDictionary? Properties { get; init; }
}
```

### 2. Executor-Level Behaviors

```csharp
public interface IExecutorBehavior
{
ValueTask HandleAsync(
ExecutorBehaviorContext context,
ExecutorBehaviorDelegate next,
CancellationToken cancellationToken);
}

public class ExecutorBehaviorContext
{
public string ExecutorId { get; init; }
public Type ExecutorType { get; init; }
public object Message { get; init; }
public Type MessageType { get; init; }
public string RunId { get; init; }
public ExecutorStage Stage { get; init; } // PreExecution or PostExecution
public IWorkflowContext WorkflowContext { get; init; }
public TraceContext TraceContext { get; init; }
public IDictionary? Properties { get; init; }
}
```

### 3. Registration API

```csharp
var workflow = new WorkflowBuilder(startExecutor)
.WithBehaviors(options =>
{
options.AddExecutorBehavior(new LoggingBehavior(logger));
options.AddExecutorBehavior(new ValidationBehavior());
options.AddWorkflowBehavior(new TelemetryBehavior());
})
.AddEdge(executor1, executor2)
.Build();
```

## Use Cases

### 1. Performance Monitoring
```csharp
public class PerformanceMonitoringBehavior : IExecutorBehavior
{
public async ValueTask HandleAsync(
ExecutorBehaviorContext context,
ExecutorBehaviorDelegate next,
CancellationToken cancellationToken)
{
var stopwatch = Stopwatch.StartNew();
try
{
return await next(cancellationToken);
}
finally
{
_metrics.RecordExecutorDuration(
context.ExecutorId,
stopwatch.ElapsedMilliseconds);
}
}
}
```

### 2. Input Validation
```csharp
public class ValidationBehavior : IExecutorBehavior
{
public async ValueTask HandleAsync(
ExecutorBehaviorContext context,
ExecutorBehaviorDelegate next,
CancellationToken cancellationToken)
{
if (context.Stage == ExecutorStage.PreExecution)
{
ValidateMessage(context.Message);
}
return await next(cancellationToken);
}
}
```

### 3. Distributed Tracing
```csharp
public class TracingBehavior : IWorkflowBehavior
{
public async ValueTask HandleAsync(
WorkflowBehaviorContext context,
WorkflowBehaviorDelegate next,
CancellationToken cancellationToken)
{
using var span = _tracer.StartSpan(
$"workflow.{context.Stage}",
new { workflowName = context.WorkflowName, runId = context.RunId });

return await next(cancellationToken);
}
}
```

### 4. Caching
```csharp
public class CachingBehavior : IExecutorBehavior
{
public async ValueTask HandleAsync(
ExecutorBehaviorContext context,
ExecutorBehaviorDelegate next,
CancellationToken cancellationToken)
{
var cacheKey = GetCacheKey(context);

if (_cache.TryGetValue(cacheKey, out var cached))
return cached;

var result = await next(cancellationToken);
_cache.Set(cacheKey, result);
return result;
}
}
```

## Benefits

### 1. **Separation of Concerns**
- Keep business logic in executors
- Extract cross-cutting concerns to behaviors
- Single Responsibility Principle

### 2. **Composability**
- Chain multiple behaviors together
- Mix and match behaviors per workflow
- Reuse behaviors across workflows

### 3. **Testability**
- Test behaviors independently
- Mock behavior pipelines in tests
- Verify behavior ordering

### 4. **Zero Breaking Changes**
- All changes are additive
- `BehaviorPipeline` property is nullable
- `WithBehaviors()` is optional
- Existing workflows work unchanged
- Fast path when no behaviors registered (zero overhead)

### 5. **Familiar Pattern**
- ASP.NET Core middleware: `app.Use()`
- MediatR: `IPipelineBehavior`
- Developers already know this pattern

## Implementation Notes

### Performance
- **Zero overhead** when no behaviors are registered (fast path check)
- **Minimal overhead** when behaviors are registered but do nothing
- Pipeline built once at workflow creation, not per execution

### Error Handling
```csharp
public class BehaviorExecutionException : Exception
{
public string BehaviorType { get; }
public string Stage { get; }

// Wraps exceptions thrown by behaviors for better diagnostics
}
```

### Execution Order
- Behaviors execute in registration order (outer to inner)
- Last registered behavior executes closest to core execution
- Similar to ASP.NET Core middleware ordering

## Comparison with Existing Patterns

| Feature | Events | Agent Middleware | **Pipeline Behaviors** |
|---------|--------|------------------|------------------------|
| Intercept execution | ❌ | ✅ | ✅ |
| Modify flow | ❌ | ✅ | ✅ |
| Workflow lifecycle | ❌ | ❌ | ✅ |
| Executor lifecycle | ❌ | ❌ | ✅ |
| Scope | Fire-and-forget | Per agent run | Per workflow/executor |
| Chain behaviors | ❌ | ✅ | ✅ |

**Note:** This complements agent middleware (`AgentRunOptions.RunMiddleware`) by providing workflow/executor-specific extension points, rather than replacing it.

## Related Issues

- #3951 - Bug about missing `AgentRunOptions.RunMiddleware` (agent-level middleware)
- #3823 - Bug about session in AG-UI middleware
- #3872 - Feature request for custom WorkflowEvent emission

This proposal addresses a **different extensibility layer** (workflow/executor behaviors) than existing agent middleware.

## Questions for Maintainers

1. **Naming**: Is "Pipeline Behavior" the right term, or would "Workflow Middleware" be clearer?
2. **Scope**: Should behaviors have access to full `IWorkflowContext`, or a limited subset?
3. **Async**: Should we support synchronous behaviors (`Func` vs `ValueTask`)?
4. **Properties**: Should `Properties` dictionary be mutable for passing data between behaviors?
5. **DI Integration**: Should behaviors support constructor injection from `IServiceProvider`?

## Proposed Implementation Plan

1. ✅ Create behavior interfaces (`IWorkflowBehavior`, `IExecutorBehavior`)
2. ✅ Create pipeline execution engine (`BehaviorPipeline`)
3. ✅ Add registration mechanism (`WorkflowBehaviorOptions`, `WithBehaviors()`)
4. ✅ Integrate into `Workflow` and `Executor` execution paths
5. ✅ Add unit tests for behavior execution and ordering
6. ⏳ Add integration tests with sample behaviors
7. ✅ Add XML documentation and code samples
8. ⏳ Update workflow documentation

## Additional Context

This feature enables enterprise-grade workflow observability and control without framework modifications. It follows established .NET patterns and maintains backward compatibility.

**Ready to implement**: POC is available and follows CONTRIBUTING.md guidelines.