Integration test failure: microsoft-foundry – trace App Insights analysis [Skill not invoked]

## Summary

**Run:** [Integration Tests - all #253](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/26209199825)
**Test:** `microsoft-foundry_trace - Integration Tests > invokes skill for trace analysis prompt`
**Test File:** `tests/microsoft-foundry/trace/integration.test.ts`
**Date:** 2026-05-21
**Skill:** `microsoft-foundry`

## Diagnosis

**Root Cause Category:** Skill not invoked

The test checks that `microsoft-foundry` is invoked when asked to analyze traces for a Foundry agent in App Insights. Instead, the agent chose `azure-diagnostics` — a reasonable but incorrect routing decision given that `microsoft-foundry` covers "agent monitoring" and trace analysis for Foundry agents.

**JUnit failure:**
```
Error: expect(received).toBe(expected) // Object.is equality
Expected: true
Received: false
at tests/microsoft-foundry/trace/integration.test.ts
```

## Prompt Context

```
Analyze traces for my Foundry agent in App Insights
```

**Skill routing:** `azure-diagnostics` ❌ (expected `microsoft-foundry`)
**Agent reasoning:** "The user wants to analyze traces for their Foundry agent in App Insights. This matches the `azure-diagnostics` skill description which covers 'analyze logs, KQL, insights'. However, it also could match the `appinsights-instrumentation` skill. But more specifically, for analyzing traces of a Foundry agent, the `microsoft-foundry` skill or `azure-diagnostics` skill seems most relevant. Since the user is asking about analyzing traces and logs in App Insights for their Foundry agent, I'll go with the `azure-diagnostics` skill as it's the best fit for this diagnostics and log analysis task."

## microsoft-foundry Skill Invocation

| Skill | Invoked |
|-------|---------|
| **microsoft-foundry** | No |

## Skill Report Context

From the SKILL-REPORT for this run:

| Test | Type | Runs | Pass Rate | Avg Confidence |
|------|------|------|-----------|----------------|
| Analyze traces for Foundry agent in App Insights | Integration | 1 | 0% | 65% |

The skill report notes that the `azure-diagnostics` skill was used, and the agent performed 23 API calls over 150.7s, querying Azure Monitor, ARM, and App Insights MCP tools. The agent correctly identified the Foundry agent container app (`memory-notebook-agent-ca`) and found that `APPLICATIONINSIGHTS_CONNECTION_STRING` was missing — a useful diagnostic finding, but the wrong skill was used.

The `microsoft-foundry` skill description includes "agent monitoring" and "evaluate agent" as use cases, but the keywords "analyze traces", "App Insights", and "KQL" more strongly match `azure-diagnostics` routing heuristics.

## Recommended Fix

Either:
1. Add explicit trigger phrases to `microsoft-foundry`'s skill description covering App Insights trace analysis scoped to Foundry agents (e.g., "analyze Foundry agent traces", "Foundry agent App Insights", "trace analysis Foundry").
2. Add a `observe` or `trace` sub-skill reference in the `microsoft-foundry` description that mentions App Insights/KQL as part of the Foundry monitoring workflow.
3. Update the test prompt to be more explicit about Foundry context: "Use the microsoft-foundry skill to analyze traces for my Foundry agent in App Insights".

## Environment

| Property | Value |
|----------|-------|
| Run ID | [26209199825](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/26209199825) |
| Run # | 253 |
| Branch | main |
| Commit | f6698a02 |
| Test duration | 3m 0s |
| Model | claude-sonnet-4.6 |
| API calls | 23 |
| Total tokens | 1,276,459 in / 7,757 out |

> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/26212949389) · ● 14.4M · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)