microsoft / microsoft/GitHub-Copilot-for-Azure

Integration test failure: azure-prepare – Timer trigger MCP missing language param [Assertion mismatch]

Open
#2,112 1 comment 0 reactions 1 assignee Claimed by @JasonYeMSFT View on GitHub
azure-prepare bug integration-test test-failure
Dominant language
Python
Stars
250
Forks
204
Avg merge
1d 12h
Merged PRs (30d)
67

Description

## Prompt

```
Create a Python Azure Functions app with a timer trigger that runs every 5 minutes and deploy to Azure.
```

## Summary

**Run:** [Integration Tests - all #201](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/25165344487)
**Test:** `azure-prepare_ - Integration Tests › functions-template-mcp › calls functions_template_get for Timer trigger`
**Result:** Fail
**Duration:** 4m 17s

## Root Cause Category

**Assertion mismatch**

## Diagnosis

### What was expected

The test checks all of the following:
1. Skill invoked ✅
2. `functions_template_get` called at least once ✅
3. Workspace files generated ✅
4. **Code indicator**: generated function code matches timer trigger pattern (`codeIndicator` regex) — **❌ FAILED** (`expect(hasCode).toBe(true)` at `tests/azure-prepare/integration.test.ts:1181`)
5. IaC files present

### What actually happened

The agent made 3 calls to `functions_template_get`:

1. **Call 1** — `{ language: "python", trigger: "timer" }` → returned a template list (successful discovery, but not a template fetch)
2. **Call 2** — `{ templateName: "timer-trigger-python-azd" }` → **error**: `Missing Required options: --language` (missing `language` parameter)
3. **Call 3** — `{ language: "python", output: "New", template: "timer-trigger-python-azd" }` → success (131KB template response)

After successfully fetching the template, the agent ran 16 bash commands to create files. However, the final workspace files did not contain the expected timer trigger code indicator pattern, causing assertion #4 to fail.

The SKILL-REPORT (Test 18) shows 4 runs with 75% pass rate (3/4), indicating this is a recurring intermittent failure tied to the parameter retry pattern and how the template content is materialized.

### Why it failed

The agent's first `functions_template_get` call fetched a template list (with `trigger: "timer"` discovery param), while the second call omitted `language` and got an error. On the third attempt the agent fetched the full template, but may have been inconsistent in how it wrote the timer trigger function body to disk — the `codeIndicator` regex did not match any code files in the workspace.

### Suggested fix

- Update the azure-prepare skill instructions to clarify the correct `functions_template_get` parameter shape: `templateName` + `language` are both required when fetching a specific template (not a list).
- Alternatively, add a guard in the skill that retries with `language` if the initial call fails with "Missing Required options: --language".

## azure-prepare Skill Invocation

| Skill | Invoked |
|-------|---------|
| **azure-prepare** | **Yes** |

## Skill Report Context

Per-test section from SKILL-REPORT.md (click to expand)

### Test 18: functions-template-mcp_Timer_trigger

**Type:** End-to-End
**Runs:** 4 | **Pass Rate:** 75% (3/4) | **Avg Confidence:** 75%

**What Happened:**
azure-functions template retrieved and generated complete Function App code with correct 5-minute timer schedule (cron: `0 */5 * * * *`). All infrastructure files (Bicep, azure.yaml) created successfully. Initial deployment encountered RBAC principal type mismatch. After 3 retries, issue resolved.

**❌ What Went Wrong:**
- First `functions_template_get` call missing `--language` parameter (error on call 2)
- Code indicator check (`hasCode`) failed — generated files did not match expected timer trigger pattern
- Required 3 tool retries to get correct template

**Skill Invocation Success Rate:** 100% (30/30 skill-invocation tests passed)
**Overall Test Pass Rate:** 98.3% (57/58) — this test is the 1 failure
**Average Confidence:** 91.7%

## Environment

- **Runner OS:** ubuntu-latest
- **Model:** claude-sonnet-4.5
- **Run URL:** https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/25165344487
- **Commit:** `070bf6454d7849a5cb0218a421ee2b61b8fcdea0`
- **Test file:** `tests/azure-prepare/integration.test.ts:1181`

**Assignees:** `@tmeschter` `@wbreza` `@kvenkatrajan` `@jongio`

> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/25177006839/agentic_workflow) · ● 3.5M · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.