Template the benchmark scripts / 模板化基准测试脚本

Assessment

This issue has not been assessed yet.

Description

Contributor guide

Open the contributing guide

First steps

1. Read the whole issue, then the project's contributing guide.
2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
3. Fork the repository and make your change on a branch.
4. Open a pull request that references the issue number.

More from SemiAnalysisAI/InferenceX

• Removal of "1 The ATOM engine is promising, however it has yet to serve production tokens. It is still in its infant stage." Open

  Difficulty 2/5 1-3 hours Newbie friendliness 68/100

  SemiAnalysisAI/InferenceX#2125 · Opened Jul 9, 2026

• [vLLM] Missing --max-model-len guard for offload-disabled agentic runs causes KV cache OOM on H100 / [vLLM] 禁用 offload 的 agentic 运行缺少 --max-model-len 保护导致 H100 上 KV 缓存 OOM Open

  Difficulty 2/5 1-3 hours Newbie friendliness 68/100

  SemiAnalysisAI/InferenceX#1587 · Opened May 29, 2026

• [experimental chore/agentx-v0.2-aiperf-testing branch] agentic build_replay_cmd: missing --tokenizer-trust-remote-code breaks Kimi-K2.5 (and other custom-code tokenizer models) / agentic build_replay_cmd 缺少 --tokenizer-trust-remote-code 导致 Kimi-K2.5 等自定义 tokenizer 模型失败 Open

  Difficulty 1/5 Under an hour Newbie friendliness 78/100

  SemiAnalysisAI/InferenceX#1369 · 3 comments · Opened May 13, 2026

• [experimental chore/agentx-v0.2-aiperf-testing branch] agentic build_replay_cmd: --cache-bust system_prefix mismatched with current aiperf scenario (requires first_turn_prefix) / agentic build_replay_cmd 的 --cache-bust system_prefix 与当前 aiperf 场景不匹配（需要 first_turn_prefix） Open

  Difficulty 1/5 1-3 hours Newbie friendliness 76/100

  SemiAnalysisAI/InferenceX#1359 · 1 comment · Opened May 13, 2026

• [Question] Is hardware contribution to the official CI cluster supported? Open

  Difficulty 5/5 Over a week Newbie friendliness 30/100

  SemiAnalysisAI/InferenceX#3122 · 3 comments · Opened Sep 15, 2026

All issues in SemiAnalysisAI/InferenceX

Similar issues

• [Bug]: notebooks/06_visualization_quickstart.ipynb fails: SHAPVisualizer.plot_waterfall raises IndexError Open

  Difficulty 2/5 1-3 hours Newbie friendliness 74/100

  bancolombia/sentinel#23 · Opened Sep 20, 2026

• test md Open
  CI

  Difficulty 2/5 1-3 hours Newbie friendliness 74/100

  sympy/sympy#30547 · 1 comment · Opened Sep 20, 2026

• quickjs: support `timeout=None` (no per-call deadline) in `CodeInterpreterMiddleware` Open
  integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature

  Difficulty 2/5 1-3 hours Newbie friendliness 74/100

  langchain-ai/deepagents#6450 · Opened Sep 20, 2026

• OpenAISamplingHandler drops ToolResultContent when a user message's list content also carries text/image blocks Open
  bug client

  Difficulty 2/5 1-3 hours Newbie friendliness 88/100

  PrefectHQ/fastmcp#5182 · 1 comment · Opened Sep 20, 2026

• last-mile drop-off on this SKILL.md Open

  Difficulty 2/5 1-3 hours Newbie friendliness 74/100

  wshobson/agents#717 · Opened Sep 20, 2026

More Python issues