Azure / Azure/azure-functions-agents-runtime

Add workflow-specific observability for dynamic workflows

Open
#80 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
9
Forks
7
Avg merge
1d 21h
Merged PRs (30d)
20

Description

> Please provide us with the following information:
> ---------------------------------------------------------------

### This issue is for a: (mark with an `x`)
```
- [ ] bug report -> please search issues before submitting
- [x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)
```

### Minimal steps to reproduce
> This is a follow-up feature request for after both of these PRs are merged:
>
> - #77 Add experimental dynamic workflows support
> - #79 feat: Integrate OpenTelemetry for enhanced observability
>
> Once dynamic workflows and runtime-owned observability both exist, add workflow-specific telemetry, logs, metrics, and failure classification so workflow execution is observable beyond the initial `start_workflow` tool call.

### Any log messages given by the failure
> N/A

### Expected/desired behavior
> Dynamic workflow execution should have first-class observability that complements the `agent.run` and `dynamic_session.execute` telemetry introduced by #79.
>
> Suggested span model:
>
> - `workflow.start` span: validation and Durable instance creation done by `start_workflow`.
> - `workflow.run` span: overall Durable orchestration execution.
> - `workflow.task` span: each workflow task, including `tool` and `wait` tasks.
>
> Suggested attributes:
>
> - `af.workflow.id`: Durable workflow instance id.
> - `af.workflow.agent_name`: agent that started the workflow.
> - `af.workflow.session_id` or hashed session prefix: correlation to the owning chat session without unnecessarily exposing raw session ids.
> - `af.workflow.status`: `Running`, `Completed`, `Failed`, `Canceled`, `Terminated`, etc.
> - `af.workflow.task_count`: total task count in the submitted plan.
> - `af.workflow.completed_count`: number of completed tasks.
> - `af.workflow.failed_task_id`: task id associated with a failure, when known.
> - `af.workflow.task.id`: workflow task id.
> - `af.workflow.task.type`: `tool` or `wait`.
> - `af.workflow.task.tool`: workflow-safe tool name for tool tasks.
> - `af.workflow.wait_duration_ms`: wait duration for wait tasks.
> - `af.workflow.cancel_requested`: whether cooperative cancel was requested.
>
> Suggested metrics:
>
> - workflow starts, completions, failures, cancellations, and terminations.
> - workflow task executions by task type/tool.
> - workflow task failures by failure kind.
> - workflow duration and task duration, if practical.
>
> Suggested failure taxonomy:
>
> - Keep `af.fault_domain` as the ownership/responsibility axis (`app`, `runtime`, `platform`, etc.).
> - Add a workflow-specific failure kind attribute such as `af.workflow.failure_kind` rather than using a single `workflow` fault domain.
> - Suggested values:
> - `plan_validation`
> - `tool_not_allowed`
> - `template_resolution`
> - `workflow_tool_failed`
> - `durable_client`
> - `orchestration_invariant`
> - Treat cooperative cancel and hard terminate as workflow statuses, not necessarily errors.
>
> Suggested structured logs:
>
> - `workflow_started`
> - `workflow_completed`
> - `workflow_failed`
> - `workflow_canceled`
> - `workflow_terminated`
> - `workflow_task_started`
> - `workflow_task_completed`
> - `workflow_task_failed`
>
> Logs should include stable identifiers such as workflow id, agent name, task id/type/tool, status, failure kind, and fault domain. Prefer a hashed session prefix over raw `session_id` where possible, since #77 already uses a hashed prefix for workflow ownership.
>
> Correlation expectations:
>
> - The initial `agent.run` span should make it clear that a workflow was started and include the `workflow_id` returned by `start_workflow` when possible.
> - Durable workflow/task telemetry should be queryable by `workflow_id` even when it runs after the original agent turn has ended.
> - If Application Insights operation correlation across the asynchronous Durable boundary is not practical in the first implementation, document the correlation story and provide KQL examples using `af.workflow.id`.

### OS and Version?
> N/A

### Versions
> Future work after #77 and #79 are merged.

### Mention any other details that might be useful
>
> This should be implemented as a separate follow-up PR rather than blocking #77 or #79. The goal is to avoid losing observability for the durable/background part of workflows, where the initial agent turn only sees `start_workflow` returning a `workflow_id` while the real work proceeds later in Durable Functions.
>
> ---------------------------------------------------------------
> Thanks! We'll be in touch soon.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing PRs #77 and #79, especially the dynamic workflow execution and OpenTelemetry changes they introduce. Map the workflow, task, metric, log, failure, and correlation requirements onto those entry points; done means background workflow execution is queryable by workflow ID with the proposed telemetry, statuses, and failure classification, or the remaining correlation limits are documented with KQL examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
backend, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.