[frontend] add selector for median or p99 for e2el, interactivity, and ttft / [frontend] 为 e2el、交互性和 ttft 添加中位数或 p99 选择器

Open

#764 opened on Feb 19, 2026 0 comments 0 reactions 1 assignee View on GitHub

@adibarra is already working on this.

Since Feb 19, 2026.

Assessment

This issue has not been assessed yet.

Description

enhancement frontend p00

Right now, we use the median data. P99 data data also exists for each of these data points. For instance, take the following artifact:

{
  "hw": "gb200",
  "conc": 4,
  "image": "lmsysorg/sglang:v0.5.8-cu130",
  "model": "deepseek-ai/DeepSeek-R1",
  "infmax_model_prefix": "dsr1",
  "framework": "dynamo-sglang",
  "precision": "fp4",
  "spec_decoding": "none",
  "disagg": true,
  "isl": 8192,
  "osl": 1024,
  "is_multinode": true,
  "prefill_tp": 4,
  "prefill_ep": 1,
  "prefill_dp_attention": "false",
  "prefill_num_workers": 1,
  "decode_tp": 4,
  "decode_ep": 1,
  "decode_dp_attention": "false",
  "decode_num_workers": 4,
  "num_prefill_gpu": 4,
  "num_decode_gpu": 16,
  "tput_per_gpu": 239.08275342140536,
  "output_tput_per_gpu": 33.241061135990364,
  "input_tput_per_gpu": 1062.4495225630653,
  "mean_ttft": 0.43425333057239185,
  "median_ttft": 0.25317291749524884,
  "std_ttft": 0.36127142112833116,
  "p99_ttft": 1.48367712290521,
  "mean_tpot": 0.006977059080877132,
  "mean_intvty": 143.32686428595977,
  "median_tpot": 0.006802988651651399,
  "median_intvty": 146.9942184538635,
  "std_tpot": 0.0005388865723481444,
  "std_intvty": 1855.6780801618415,
  "p99_tpot": 0.008094592967696608,
  "p99_intvty": 123.53925688304984,
  "mean_itl": 0.34069569860559523,
  "median_itl": 0.33583703998010606,
  "std_itl": 0.06460103068039673,
  "p99_itl": 0.39620181820704603,
  "mean_e2el": 6.8478497144227735,
  "median_e2el": 6.729302968495176,
  "std_e2el": 0.8481532815896575,
  "p99_e2el": 8.62211772535491
}

Users should have the option to select between SLA (median or P99 for now). P99 should be the default.

中文说明

前端为 e2el、交互性和 TTFT 添加中位数或 p99 选择器。

Dominant language: Python
Stars: 1.7k
Forks: 303
Avg merge: 1d 13h
Merged PRs (30d): 284

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.