microsoft / microsoft/Agent365-Samples

Agent 365 observability: traces reach CloudAppEvents but Activity tab stays empty in Agent 365

Open
#318 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
108
Forks
59
PR merge metrics
No merged PRs in 30d

Description

What's happening

I built a Python agent using the sample in this repo (python/agent-framework/sample-agent)
and added Agent 365 observability to it. The telemetry is reaching Agent 365 — I can see
the traces in Microsoft Defender (Advanced Hunting, CloudAppEvents table) a few minutes
after I chat with the agent.

The problem: the same traces do NOT show up in the Microsoft 365 admin center, under
Agents → my agent → Activity tab. That tab stays empty and says "Nothing to show"
(Total sessions: 0), even though the traces are clearly in Defender.

My setup

  • Tenant has Microsoft Agent 365 and Microsoft 365 E5 licenses, both assigned to a user
    (not just owned by the tenant)
  • I created the blueprint and agent identity by running a365 setup all. It granted the
    Observability OtelWrite permission to the blueprint, and the agent identity has its
    federated identity credential.
  • Blueprint app ID: 3206eb8b-1902-448f-84d2-45bc8e30ec51
  • Agent identity app ID: 08165898-aac2-42b3-81fd-a1809f25cbd5
  • I'm running the agent locally (not deployed) and testing it with the M365 Agents Playground
  • I'm using the S2S observability path. The agent gets its observability token through the
    3-step FMI chain (blueprint secret → blueprint federated credential → agent identity →
    observability token). The token is acquired successfully every time.

What works

  • The agent exports telemetry with no errors. It posts successfully to
    agent365.svc.cloud.microsoft.
  • In Defender's CloudAppEvents, I see InvokeAgent and InferenceCall rows.
  • On each record, these fields are correct:
    • TargetAgentId = 08165898-aac2-42b3-81fd-a1809f25cbd5 (my agent identity)
    • TargetAgentBlueprintID = 3206eb8b-1902-448f-84d2-45bc8e30ec51 (my blueprint)

The issue

On those same records, the top-level identity fields are all zeros:

  • AgentId = 00000000-0000-0000-0000-000000000000
  • AgentBlueprintId = 00000000-0000-0000-0000-000000000000
  • UserId = N/A
  • UserKey = 00000000-0000-0000-0000-000000000000

So the traces are in Defender, but the admin center Activity tab shows nothing.

My questions

  1. What fills in the top-level AgentId and AgentBlueprintId fields? They look like they
    come from the Agent 365 backend, not from my agent's telemetry — because the
    TargetAgentId and TargetAgentBlueprintID (which come from my telemetry) are correct,
    but the top-level ones are zeros. The documentation doesn't explain the difference.

  2. Does the admin center Activity tab use the top-level AgentId/AgentBlueprintId (the
    zeroed ones) to find an agent's activity, instead of the TargetAgentId? If so, what do
    I need to do to get those filled in?

  3. Should the Activity tab show data for an agent that's running locally and tested through
    the Agents Playground? Or does it only work for a deployed agent that's invoked by a
    real user (for example, from Teams or email)?

  4. I currently have two registrations for the same agent (both named "eppcgovagent Agent"),
    because I ran a365 setup all twice. Could a duplicate or stale registration be the
    reason the backend can't link my traces to the agent?

I'm happy to share the full telemetry output, my export logs, or do a screen share.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with python/agent-framework/sample-agent and the a365 setup all entry point, then review the S2S observability export and its CloudAppEvents records. Compare the TargetAgent and top-level identity fields while testing through the M365 Agents Playground. Done means identifying whether local testing, duplicate registrations, or backend identity mapping explains the empty Activity tab and documenting the required setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.