Azure / Azure/azure-sdk-tools

[Epic][TypeSpec Authoring][Benchmark] Add advanced ARM benchmark cases

Open
#15,725 0 comments 0 reactions 0 assignees View on GitHub
AzSDK Tools Agent Epic
Dominant language
C#
Stars
135
Forks
260
Avg merge
3d 1h
Merged PRs (30d)
143

Description

Add advanced ARM benchmark cases for TypeSpec authoring (e.g., complex resource scenarios, custom actions, LROs, child/extension resources, advanced versioning, polymorphism, etc.).

inner loop benchmark(including evaluate skill, mcp tool etc):
https://gist.github.com/jeo02/99b9b441b10b383b6d9bbdcfd17a4a70
framework Repo: https://github.com/microsoft/evaluate
Quickstart: [Get Started | evaluate](https://literate-engine-r3wnl4v.pages.github.io/get-started/)
pr on authoring skill benchmark: https://github.com/Azure/azure-sdk-tools/pull/15190 should be able to run locally

Contributor guide

Open the contributing guide

Research direction

Start with the linked inner-loop benchmark gist and the evaluate framework quickstart, then review the authoring skill benchmark PR #15190. Identify how benchmark cases are structured and run locally through evaluate. Done means advanced ARM authoring scenarios, including complex resources, custom actions, LROs, versioning, and polymorphism, are covered and runnable.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
devtools, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.