microsoft / microsoft/agent-framework

.NET: Per-executor chat-client forwarding hook for InProcessExecution workflows (observability)

Open
#6,918 1 comment 0 reactions 1 assignee View on GitHub

@peibekwe is already working on this.

Since Jul 7, 2026.

.NET workflows
Dominant language
Python
Stars
13.6k
Forks
2.3k
Avg merge
2d 45m
Merged PRs (30d)
358

Description

### Summary

When running a multi-agent workflow via `InProcessExecution`, there is currently no supported way to observe each executor's **chat-boundary** activity (the `ChatClientAgent` request/response at the model interface) from outside the executor. An observer can see the workflow-level events and each executor's *reported* output, but not the per-executor chat turns (system prompt, real token usage, finish reason, tool calls) as the model actually saw them.

### Motivation

I am building evaluation tooling ([AgentEval](https://github.com/AgentEvalHQ/AgentEval)) that reconciles a framework's self-reported per-executor ledger (summed tokens + finish reason) against chat-boundary truth, to detect silent divergences — hidden retries, token under-reporting, suppressed finish reasons, provider safety interventions. This works today for a single `ChatClientAgent` (I wrap its `IChatClient`) and for pre-wired / replayed traces, but **not** for executors running inside a live `InProcessExecution` workflow, because the workflow constructs/owns the executors' chat clients and does not surface a seam to intercept or forward per-executor chat activity.

### Proposal (any one of these would unblock us)
1. **A per-executor chat-client factory / decorator hook** on the workflow builder — e.g. an `Func` applied when each `ChatClientAgent` executor is materialized, so observers can wrap the client and record chat-boundary entries per executor.
2. **A per-executor chat event** on the workflow event stream carrying the request/response (or a correlation id back to the owning executor), analogous to the existing executor output events.
3. **A documented extension point** to attach middleware to each executor's chat pipeline at build time.

### What I do today (workaround)
For direct agents I wrap `IChatClient`; for workflows I fall back to replay / pre-wired per-executor traces and clearly report untraced executors as `NoTruth` (excluded from the fidelity aggregate). A live hook would let the reconciliation run on real in-flight workflow executions.

### References

- Related: #3075 (.NET: Workflow Middleware to Support Customizing Telemetry) — a per-executor chat seam would also serve that scenario.
- Downstream consumer: AgentEval Glass Box `workflow-trace-fidelity` benchmark.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.