Azure / Azure/azure-sdk-tools

[Epic][TypeSpec Authoring][Telemetry] Collect user feedback from early adopters and telemetry analysis

Open
#14,902 1 comment 0 reactions 2 assignees Claimed by @josefree View on GitHub
AzSDK Tools Agent Epic
Dominant language
C#
Stars
135
Forks
260
Avg merge
1d 22h
Merged PRs (30d)
115

Description

## Overview

Collect user feedback for the **azure-typespec-author** Copilot skill to measure adoption, effectiveness, and identify areas for improvement.

## Feedback Channels

### 1. Skill Invocation Telemetry
- Number of skill invocations (total and per-user)
- Task completion and failure rates
- Task type breakdown (API Versioning, Resource Definitions, Resource Operations, Models & Types)
- Pending on dashboard

### 2. Sanitized User Prompts Telemetry
- Collect sanitized/anonymized user prompts to understand common usage patterns
- Identify most/least used prompt categories
- Detect unclear or failing prompt patterns

### 3. User Conversations
- Conduct feedback sessions with early adopter teams
- Gather qualitative insights on authoring experience, output quality, and pain points

### 4. User Quick Feedback
- Collect in-session thumbs up/down or rating feedback after task completion
- Track user satisfaction trends over time

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.