Azure / Azure/azure-sdk-tools

[TypeSpec Authoring][Telemetry] Conduct user study with early adopters

Open
#15,741 2 comments 0 reactions 1 assignee Claimed by @josefree View on GitHub
AzSDK Tools Agent
Dominant language
C#
Stars
135
Forks
260
Avg merge
3d 1h
Merged PRs (30d)
143

Description

## Summary

Conduct user study sessions with early adopters of the **azure-typespec-author** Copilot skill to gather qualitative feedback on authoring experience, output quality, and pain points.

Parent epic: #14902 (Channel #3 — User Conversations).

## Goals

- Validate whether the skill is solving real authoring problems for early adopters.
- Identify the top pain points and friction in the end-to-end flow (skill invocation → plan → generation → validation).
- Surface unmet needs and prioritize backlog items based on real user voice.
- Capture concrete prompts / scenarios that work well and ones that don't, to feed back into benchmark cases.

## Scope

1. **Recruit participants**
- Identify 5–8 early adopter teams / individuals currently authoring TypeSpec specs (ARM and data-plane).
- Mix of new authors and experienced authors.

2. **Prepare study materials**
- Interview script / question bank (open-ended + targeted).
- Optional task scenarios for think-aloud sessions (e.g., add a new API version, add a new resource, add a custom action).
- Consent / recording policy.

3. **Run sessions**
- 30–45 min each: 1:1 interview and/or observed task session.
- Capture: usage frequency, common prompts, output quality perception, blockers, workarounds, feature requests.

4. **Synthesize findings**
- Affinity-grouped themes, severity, frequency.
- Map each finding to existing epics / issues or file new ones.
- Share a readout with the team.

## Deliverables

- [ ] Interview script and task scenarios.
- [ ] Notes / recordings from each session (stored per privacy policy).
- [ ] Findings report with prioritized themes and action items.
- [ ] Backlog items filed (linked back to this issue) for the top issues raised.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.