Azure / Azure/azure-sdk-tools

[TypeSpec Authoring] Conduct user study for the initial assessment skill

Open
#16,992 0 comments 0 reactions 0 assignees View on GitHub
AzSDK Tools Agent dev inner loop needs-triage
Dominant language
C#
Stars
135
Forks
260
Avg merge
1d 22h
Merged PRs (30d)
115

Description

## Parent issue

Part of #14529.

## Goal

Work with Josephine to run a user study of the published initial TypeSpec assessment skill and learn from users' feedback.

## Scope

- Define the study protocol, target participants, representative TypeSpec changes, and success criteria with Josephine.
- Ask participants to exercise the initial version using realistic assessment scenarios.
- Observe task completion and record confusion points, unmet expectations, latency perception, missing findings, and unsupported findings.
- Synthesize qualitative and quantitative feedback into prioritized improvements.
- Link follow-up issues to this task and #14529.

## Acceptance criteria

- The study plan and participant profile are documented before sessions begin.
- Participants exercise the published initial version on representative TypeSpec changes.
- Feedback and observations are documented with identifying information removed as appropriate.
- Results include prioritized recommendations and linked follow-up tasks.

Contributor guide

Open the contributing guide

Research direction

Start with the published initial TypeSpec assessment skill and parent issue #14529, then work with Josephine to define the study protocol, participant profile, representative TypeSpec changes, and success criteria. Run the sessions, document anonymized observations and feedback, and consider the work done when prioritized recommendations and linked follow-up tasks are recorded.

Written by the indexing model from the issue text.

Assessment

Domain
developer-experience, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.