Azure / Azure/azure-functions-agents-runtime

Add startup diagnostics summary for resolved agents and capabilities

Open
#39 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
9
Forks
7
Avg merge
1d 21h
Merged PRs (30d)
20

Description

## Problem

The runtime currently logs useful pieces of startup and registration state, but not a consolidated summary of the final resolved app configuration.

Today we log fragments such as:

- user tool discovery count from `tools/`
- MCP server load count or invalid-entry warnings
- skill discovery count
- normal trigger registration
- HTTP route registration
- debug chat/API/MCP endpoint registration

However, users do not get an easy-to-scan startup summary that answers:

- Which agent files were loaded?
- Which Azure Function name, route, slug, and debug surfaces did each agent register?
- Which model and timeout did each agent resolve to?
- Which user tools, MCP servers, skills, and system tools are enabled for each agent?
- Which excludes were applied?
- Is the code interpreter / ACA Dynamic Sessions system tool enabled or disabled?

Without this, tool/capability confusion is diagnosed by reading scattered logs or waiting until runtime failures.

## Proposal

Add a concise startup diagnostics summary after each agent is resolved/validated/capabilities are built, plus a final app-level summary.

This should be log-based rather than a new HTTP/debug endpoint for now. Startup logs solve most discoverability issues with less API surface area.

## Suggested per-agent log content

For each resolved agent, log a structured or consistently formatted info-level entry containing:

- source file
- display name
- resolved Azure Function name, when applicable
- trigger type and route/methods, when applicable
- debug endpoint surfaces and routes/tool name, when applicable
- resolved model/provider information where available without leaking secrets
- timeout
- enabled user tool names
- enabled MCP server names
- enabled skill names
- system tool status, including code interpreter enabled/disabled and endpoint configured/unconfigured without logging secrets
- applied exclude lists

## Suggested final summary

At the end of `create_function_app()`, log totals such as:

- number of agent files loaded
- number of normal triggers registered
- number of HTTP agent routes registered
- number of debug chat/API/MCP surfaces registered
- number of discovered user tools, MCP servers, and skills
- number of agents with code interpreter enabled

## Safety / logging guidance

- Do not log secrets, API keys, bearer tokens, connection strings, or raw headers.
- For URLs/endpoints, log either the host or a redacted/stable summary if full URLs may contain sensitive data.
- Keep logs useful at `INFO`; reserve noisy per-file detail for `DEBUG` if needed.
- Prefer stable names and counts over object reprs.

## Relationship to existing logs

This should build on existing discovery/registration logs rather than replacing every message. The goal is a single summary users can find in Functions startup logs when asking, "what did the runtime actually register and enable?"

## Required docs and sample updates

- Add a README troubleshooting/debugging note that points users to the startup diagnostics summary.
- Update `docs/architecture.md` if needed to mention where the summary is emitted in the startup pipeline.
- Add tests that capture/log-assert the important fields without depending on exact formatting too tightly.

## Acceptance criteria

- Startup logs include one concise resolved summary per agent.
- Startup logs include an app-level summary after registration completes.
- The summary includes enabled capabilities by name: user tools, MCP servers, skills, and system tools.
- The summary includes endpoint/route/debug surface information without leaking secrets.
- Docs tell users where to look for this information when debugging missing tools or endpoints.

Contributor guide

Open the contributing guide

Research direction

Start by tracing create_function_app() and the existing discovery and registration logs to identify where resolved agent capabilities and final registration totals are available. Review the README troubleshooting/debugging note and docs/architecture.md requirements, then add focused log-capture tests that verify important fields without relying on exact formatting; done means both per-agent and app-level summaries are emitted without secrets.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
backend, documentation, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.