Measure and improve Ask Mantosh perceived latency and failure recovery

Priority: Medium

Problem

Ask Mantosh streams answers and has explicit retry/failure handling, but there is no product-level latency baseline showing what a visitor actually experiences from click to usable answer. Slow first response, cold paths, provider delays, or unclear recovery can weaken trust even when answer quality is high.

Goal

Measure the end-to-end Ask Mantosh experience and improve perceived responsiveness without weakening retrieval, evidence, privacy, or safety controls.

Scope

• Measure launcher-to-ready time, request-to-first-visible-response time, time to complete answer, and failure/retry paths in production.
• Separate deterministic responses, cached responses, retrieval/generation responses, and cold/runtime failure cases.
• Identify the dominant latency contributors before optimizing.
• Review loading/thinking states, streaming readability, retry wording, and what happens when quota/provider/network failures occur.
• Prefer caching, routing, payload, UX, or infrastructure improvements that preserve evidence quality.
• Define realistic latency/error budgets rather than absolute speed targets unsupported by the provider stack.

Acceptance criteria

• Production latency baselines exist for representative Ask Mantosh paths.
• First-visible-response and full-answer timings are recorded separately.
• Failure/retry experience is tested under controlled provider/network/quota failures.
• Material latency or recovery defects are fixed or split into focused issues.
• Any optimization preserves the evidence-first and privacy boundaries.
• Release/operations documentation records the resulting latency and failure expectations.

Start by tracing the Ask Mantosh launcher, streaming response, and retry/failure paths, then identify where production timings and failure outcomes can be recorded. Separate deterministic, cached, retrieval/generation, and cold/runtime cases before proposing changes. Done means representative latency baselines, controlled recovery tests, documented expectations, and focused follow-up issues for material defects.