microsoft / microsoft/mcp

Add tool to fetch latest Azure SDK versions per language

Open
#2,486 1 comment 1 reaction 1 assignee Claimed by @g2vinay View on GitHub
enhancement needs-team-attention server-Azure.Mcp tools-Core
Dominant language
C#
Stars
3.7k
Forks
624
Avg merge
2d 20h
Merged PRs (30d)
220

Description

## Problem

LLMs consistently generate code using **outdated Azure SDK package versions** across multiple languages (Java, Python, JavaScript), regardless of configuration (baseline, skills, or Azure MCP).

### Evidence from hyoka Evaluation

From the [hyoka evaluation results](https://gist.github.com/samvaity/42ac2636028e25acb9f9c2900b960cac#4-universal-weaknesses-across-all-configs):

> **Latest Package Versions**: 9/9 ❌
> All configs generate outdated SDK versions (e.g., `azure-storage-blob 12.28.0` vs latest `12.30.0`)

This pattern appears across:
- ❌ Baseline (no tools, no skills)
- ❌ Baseline with azure-sdk-java plugin skills
- ❌ Azure MCP server

### Root Cause

LLMs are trained months prior to current SDK releases. When generating code, they reference SDK versions that were "latest" during their training cutoff, which are now outdated.

**This affects:**
- Users missing critical hotfixes
- Users missing new features and performance improvements
- Potential security vulnerabilities from outdated packages
- Overall developer experience (outdated examples in generated code)

## Proposed Solutions

### Option 1: MCP Tool to Fetch Latest SDK Versions

Add a tool to the Azure MCP server that can fetch the latest SDK version for a given service and language.

**Usage from skills:**
```markdown
# Use the Azure SDK Version tool to get the latest package
# Query: get_latest_sdk_version(service="storage", language="java")
# This returns: com.azure:azure-storage-blob:12.30.0
```

**Pros:**
- Centralized, reusable across all skills
- Can be called explicitly from skill content
- Works even without skills (baseline + Azure MCP)

**Cons:**
- Adds another tool to the MCP server
- LLM may not always choose to call it
- Needs maintenance as package managers/registries change

### Option 2: Python Code in Skills

Embed Python code directly in skill content that processes SDK release information from GitHub.

**Implementation:**
- Process `CHANGELOG.md` or release notes from Azure SDK repos
- Generate a curated version list file in a clean format
- Skills reference this file for latest versions

**Pros:**
- No new MCP tool needed
- Skills team owns the solution
- Can include additional metadata (release notes, breaking changes)

**Cons:**
- Rate limiting when hitting GitHub API
- Duplicate logic across language-specific skills
- Skills need to be regenerated when versions update

### Option 3: Enhanced Skill Content with Version Guidance

Update skills to reference SDK README files or documentation pages that list current versions.

**Example:**
```markdown
# For latest versions, check:
# - Python: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/storage/azure-storage-blob/README.md
# - Java: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/storage/azure-storage-blob/README.md
# - .NET: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/README.md
```

**Pros:**
- No new infrastructure needed
- Points LLM to authoritative sources
- Works with existing web_fetch capabilities

**Cons:**
- LLM may not follow the guidance
- Adds latency to fetch README content
- No guarantee the LLM will parse version correctly

## Recommendation

**Start with Option 1 (MCP Tool)** for immediate impact, with Option 3 (Skill Content) as a complementary approach?

## Open Questions

1. **Skill Integration**: Should skills explicitly instruct to call this tool, or rely on tool description?
2. **Caching**: How often should version data be refreshed? (Daily? Weekly?)
3. **Scope**: Should this handle all Azure packages or start with a subset (e.g., top 10 services)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.