Owner

SemiAnalysisAI

3 indexed repositories · View on GitHub

InferenceX
Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200...

Python · 1736 stars
InferenceX-app
Dashboard for InferenceX™, Open Source Continuous Inference | InferenceX 仪表板

TypeScript · 46 stars
ClusterMAX
tooling to support clustermax.ai, the industry standard rating system for neoclouds

Python · 24 stars

Beginner friendly Everything

Open All

[Ingest] guard against coverage regressions when a narrow sweep supersedes a published curve / 防止窄矩阵 sweep 替换已发布曲线导致覆盖面退化 Open

Difficulty 5/5 Over a week Newbie friendliness 50/100

SemiAnalysisAI/InferenceX-app#1185 · Opened Sep 20, 2026
Read-only k8s audit aborts at check 4/41 with a jq parse error Open

Difficulty 1/5 1-3 hours Newbie friendliness 88/100

SemiAnalysisAI/ClusterMAX#21 · Opened Sep 18, 2026
[Question] Is hardware contribution to the official CI cluster supported? Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

SemiAnalysisAI/InferenceX#3122 · 3 comments · Opened Sep 15, 2026
Runtime JIT compilation contaminates published TTFT and TPOT measurements Open

Difficulty 5/5 Over a week Newbie friendliness 28/100

SemiAnalysisAI/InferenceX#2819 · Opened Sep 3, 2026
Single-node num_gpus ignores --data-parallel-size, inflating tput_per_gpu Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

SemiAnalysisAI/InferenceX#2714 · Opened Aug 24, 2026
[Power] Power measurement roadmap Open

SemiAnalysisAI/InferenceX#2681 · 1 assignee · Opened Aug 19, 2026
[CI] Recipe bring-up for /overview combos with no data / [CI] 缺数据 combo 的 recipe bring-up 跟踪 Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

SemiAnalysisAI/InferenceX#2587 · Opened Aug 12, 2026
[WIP] refactor/cleanup repo Open

SemiAnalysisAI/InferenceX#2382 · 2 assignees · Opened Jul 28, 2026
Proposal: add Unofficial DeepSeek-V4-Flash 4xH200 SGLang recipe / 提案：新增非官方 DeepSeek-V4-Flash 4×H200 SGLang 配方 Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

SemiAnalysisAI/InferenceX#2315 · 2 comments · Opened Jul 23, 2026
[CI] Same-batch comparability for `/overview` data / `/overview` 数据的同批次可比性 Open

SemiAnalysisAI/InferenceX#2304 · 1 assignee · Opened Jul 22, 2026
[Feat] Reduce cognitive load in `/inference` dashboards / 降低 `/inference` 仪表盘的认知负担 Open

SemiAnalysisAI/InferenceX-app#615 · 1 assignee · Opened Jul 22, 2026
请问是否有长序列的比较维度？ Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

SemiAnalysisAI/InferenceX#2287 · 12 comments · Opened Jul 20, 2026
measuring power Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

SemiAnalysisAI/InferenceX#2250 · Opened Jul 16, 2026
Upstream SGLang fix for disaggregated prefill TP-rank desync / 向 SGLang 上游提交分离式预填充 TP rank 不同步修复 Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

SemiAnalysisAI/InferenceX#2151 · Opened Jul 10, 2026
Clean up configuration ownership and redundant defaults after PR #2127 / 清理 PR #2127 后的配置归属与冗余默认值 Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

SemiAnalysisAI/InferenceX#2150 · Opened Jul 10, 2026
Removal of "1 The ATOM engine is promising, however it has yet to serve production tokens. It is still in its infant stage." Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

SemiAnalysisAI/InferenceX#2125 · Opened Jul 9, 2026
Refactor common result metadata processing into a shared helper Open

SemiAnalysisAI/InferenceX#2105 · 1 comment · 1 assignee · Opened Jul 6, 2026
Add gate on full-sweep / fail-fast runs to prevent wasted compute Open

dev velocity enhancement p1

SemiAnalysisAI/InferenceX#2103 · 2 assignees · Opened Jul 6, 2026
runners.yaml: add per-cluster directory mappings (centralize paths hardcoded in launch_*.sh) / runners.yaml：添加集群目录映射（集中管理 launch_*.sh 中硬编码的路径） Open

SemiAnalysisAI/InferenceX#1973 · 1 assignee · Opened Jul 1, 2026
Add local "dry run" mode: run all GHA input/script-generation logic without triggering a run / 添加本地「试运行」模式：执行所有 GHA 输入/脚本生成逻辑而不触发实际运行 Open

SemiAnalysisAI/InferenceX#1972 · 1 assignee · Opened Jul 1, 2026
Generalize per-subpoint benchmark checkpointing for reruns / 泛化基准测试子点检查点机制以支持重跑 Open

SemiAnalysisAI/InferenceX#1971 · 1 assignee · Opened Jul 1, 2026
Template the benchmark scripts / 模板化基准测试脚本 Open

SemiAnalysisAI/InferenceX#1945 · 1 assignee · Opened Jun 26, 2026
Quality Evals that stress longer decode / 压力测试长解码的质量评估 Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

SemiAnalysisAI/InferenceX#1920 · 2 comments · Opened Jun 24, 2026
Front End: Select Date Range too often / 前端：日期范围选择过于频繁 Open

Difficulty 3/5 1-2 days Newbie friendliness 58/100

SemiAnalysisAI/InferenceX#1919 · 1 comment · Opened Jun 24, 2026
[enhancement] add more info to runners yaml / [enhancement] 在 runners yaml 中添加更多信息 Open

SemiAnalysisAI/InferenceX#1910 · 1 assignee · Opened Jun 23, 2026
[enhancement] multiple conc against same server / [enhancement] 同一服务实例运行多个并发度测试 Open

SemiAnalysisAI/InferenceX#1909 · 1 assignee · Opened Jun 23, 2026
Publish VLLM and SGLANG configs where possible / 尽可能同时发布 vLLM 和 SGLang 配置 Open

Difficulty 5/5 Over a week Newbie friendliness 32/100

SemiAnalysisAI/InferenceX#1889 · 2 comments · Opened Jun 23, 2026
[codex discussion]: b200 b300 agentx dsv4 vllm and sglang / [codex discussion]: B200 B300 AgentX DSV4 的 vLLM 与 SGLang 方案讨论 Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

SemiAnalysisAI/InferenceX#1880 · Opened Jun 21, 2026
M3 mi325 profile run / MiniMax-M3 MI325X 性能剖析运行 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

SemiAnalysisAI/InferenceX#1765 · 1 comment · Opened Jun 15, 2026
Canary only considers one config key / 金丝雀门控仅考虑一个配置键 Open

SemiAnalysisAI/InferenceX#1740 · 1 assignee · Opened Jun 13, 2026
[Reproducibility] dsr1 0528 dynamo + mtp on b200 / [Reproducibility] DSR1 0528 Dynamo + MTP 在 B200 上的复现 Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

SemiAnalysisAI/InferenceX#1691 · 9 comments · Opened Jun 9, 2026
aiperf: Weka synth buffer should reset context when truncation deletes emitted segments / aiperf：Weka 合成缓冲区截断删除已发出段时应重置上下文 Open

Difficulty 3/5 1-2 days Newbie friendliness 70/100

SemiAnalysisAI/InferenceX#1665 · Opened Jun 4, 2026
[Tracking Issue] Synthetic Acceptance for MTP Benchmarks / [Tracking Issue] MTP 基准测试的合成接受长度 Open

in progress p0

SemiAnalysisAI/InferenceX#1651 · 7 comments · 1 assignee · Opened Jun 2, 2026
Search optimizations (Not SEO) / 搜索优化（非 SEO） Open

p3

Difficulty 3/5 1-2 days Newbie friendliness 55/100

SemiAnalysisAI/InferenceX-app#406 · Opened May 29, 2026
[vLLM] Missing --max-model-len guard for offload-disabled agentic runs causes KV cache OOM on H100 / [vLLM] 禁用 offload 的 agentic 运行缺少 --max-model-len 保护导致 H100 上 KV 缓存 OOM Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

SemiAnalysisAI/InferenceX#1587 · Opened May 29, 2026
[Reproducibility] Request DeepSeek-R1 H100 2P2D run artifacts and network inventory / [Reproducibility] 请求 DeepSeek-R1 H100 2P2D 运行产物和网络清单 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

SemiAnalysisAI/InferenceX#1580 · 2 comments · Opened May 28, 2026
[AI Generated] [Handoff] out of 70+ image updates, 13 stuck Klaud Cold PRs need upstream coordination / scope decisions / [AI Generated] [Handoff] 70 多个镜像更新中 13 个 Klaud Cold PR 卡住，需上游协调或范围决策 Open

SemiAnalysisAI/InferenceX#1511 · 1 assignee · Opened May 18, 2026
Inference engine startup included in GPU metric capture / 推理引擎启动阶段被纳入 GPU 指标采集 Open

SemiAnalysisAI/InferenceX#1507 · 2 comments · 2 reactions · 1 assignee · Opened May 18, 2026
[infra] h200-nb_0 / h200-nb_1: enroot /mnt/image-storage full — can't unpack TRT-LLM v1.3.0rc14 image / [infra] h200-nb_0 / h200-nb_1：enroot /mnt/image-storage 已满，无法解压 TRT-LLM v1.3.0rc14 镜像 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

SemiAnalysisAI/InferenceX#1496 · 1 comment · Opened May 18, 2026
[Klaud Cold] sglang v0.5.12 DeepGemm regression on B300: CUDA_ERROR_ILLEGAL_ADDRESS in fp8_fp4_gemm_nt TMA descriptor init / [Klaud Cold] SGLang v0.5.12 B300 上 DeepGemm 回归：fp8_fp4_gemm_nt TMA 描述符初始化中 CUDA_ERROR_ILLEGAL_ADDRESS Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

SemiAnalysisAI/InferenceX#1463 · 1 comment · Opened May 18, 2026
Better evals page sharing / 改进评估页面的分享功能 Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

SemiAnalysisAI/InferenceX-app#357 · Opened May 17, 2026
[experimental chore/agentx-v0.2-aiperf-testing branch] agentic build_replay_cmd: missing --tokenizer-trust-remote-code breaks Kimi-K2.5 (and other custom-code tokenizer models) / agentic build_replay_cmd 缺少 --tokenizer-trust-remote-code 导致 Kimi-K2.5 等自定义 tokenizer 模型失败 Open

Difficulty 1/5 Under an hour Newbie friendliness 78/100

SemiAnalysisAI/InferenceX#1369 · 3 comments · Opened May 13, 2026
[experimental chore/agentx-v0.2-aiperf-testing branch] agentic build_replay_cmd: --cache-bust system_prefix mismatched with current aiperf scenario (requires first_turn_prefix) / agentic build_replay_cmd 的 --cache-bust system_prefix 与当前 aiperf 场景不匹配（需要 first_turn_prefix） Open

Difficulty 1/5 1-3 hours Newbie friendliness 76/100

SemiAnalysisAI/InferenceX#1359 · 1 comment · Opened May 13, 2026
Packaging and versioning for benchmarking tools and utilities / 基准测试工具和实用程序的打包与版本管理 Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

SemiAnalysisAI/InferenceX#1302 · Opened May 8, 2026
New ISL/OSL / 新增 ISL/OSL 组合 Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

SemiAnalysisAI/InferenceX-app#309 · 2 comments · Opened May 5, 2026
mirror to gitcode / 镜像到 gitcode Open

p2

SemiAnalysisAI/InferenceX#1281 · 2 assignees · Opened May 5, 2026
Historical view redesign / 历史数据视图重新设计 Open

diff/hard p1

Difficulty 5/5 Over a week Newbie friendliness 35/100

SemiAnalysisAI/InferenceX-app#273 · 5 comments · Opened Apr 30, 2026
Reproduce drawer / 复现抽屉 Open

diff/hard p2

Difficulty 5/5 Over a week Newbie friendliness 45/100

SemiAnalysisAI/InferenceX-app#270 · 5 comments · Opened Apr 30, 2026
Embeddable charts / 可嵌入图表 Open

diff/medium p1

Difficulty 5/5 Over a week Newbie friendliness 35/100

SemiAnalysisAI/InferenceX-app#269 · 5 comments · Opened Apr 30, 2026
[Auto] Docker Image Updates Available - 2026-04-25 / Docker 镜像更新可用 - 2026-04-25 Open

claude-task docker-update

Difficulty 3/5 1-2 days Newbie friendliness 52/100

SemiAnalysisAI/InferenceX#1154 · 18 comments · Opened Apr 25, 2026

Showing the newest 100

This page lists what was indexed most recently. Advanced filter has the whole inventory, narrowed by language, difficulty and how long a task takes.

Open advanced filter

SemiAnalysisAI

Showing the newest 100

Get new issues in your inbox