[TypeSpec Authoring] Reduce assessment skill latency and define performance budgets
- Dominant language
- C#
- Stars
- 135
- Forks
- 260
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 143
Description
## Parent issue
Part of #14529.
## Problem
A full assessment currently takes about eight minutes on average, which is too slow for an effective inner-loop experience.
## Goal
Measure and reduce end-to-end assessment latency without reducing finding quality, evidence requirements, or deterministic failure behavior.
## Scope
- Instrument preparation, tool execution, model judgment, rendering, and total duration for complete and fast modes.
- Establish representative repositories and change sets for repeatable performance measurement.
- Identify critical-path bottlenecks and unnecessary serial work, repeated analysis, oversized context, and redundant tool calls.
- Parallelize independent work and cache or reuse deterministic evidence where correctness permits.
- Define latency budgets and regression thresholds for complete and fast modes.
- Track quality and latency together so optimizations cannot pass by dropping findings or evidence.
## Acceptance criteria
- A checked-in or CI-accessible performance baseline records median and tail latency by phase and mode.
- Complete-mode average latency improves materially from the current approximately eight-minute baseline.
- Fast mode has a documented, measurably lower latency budget than complete mode.
- Benchmark quality and finding parity requirements continue to pass.
- Performance regressions are visible in benchmark or CI output.
Contributor guide
Research direction
Start by locating the assessment pipeline entry points for preparation, tool execution, model judgment, rendering, and complete or fast modes, then find the existing benchmark or CI output. Establish representative repositories and change sets for repeatable measurements. Done means phase and total latency baselines, budgets, regression visibility, and quality or finding-parity checks are recorded and passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100