modelscope / modelscope/ms-swift

[Feature] swift deploy 增加可观测性指标暴露(QPS/延时/tokens)与 Prometheus 集成

Open
#8,230 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement stale
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

Checklist / 检查清单
  • I have searched existing issues, and this is a new feature request. / 我已经搜索过现有的 issues,确认这是一个新的 Feature Request。
Feature Request Description / Feature Request 描述
问题背景

目前 swift deploy 已支持基础健康检查(如 health/ping)以及日志侧统计(如 log_interval 打印 tokens/s),但缺少类似 vLLM server 的标准化监控指标暴露能力,不利于生产环境容量规划、SLO 监控和告警。

期望能力

希望 swift deploy 增加内置 metrics(建议 Prometheus 格式)并提供标准抓取端点,例如 /metrics

建议包含的核心指标:

  1. 吞吐与请求
  • swift_requests_total{model,route,status}
  • swift_request_qps(或由 total + rate 计算)
  • swift_tokens_input_total{model}
  • swift_tokens_output_total{model}
  • swift_tokens_per_second{model}
  1. 时延
  • swift_request_latency_seconds_bucket{model,route}(Histogram)
  • swift_queue_latency_seconds_bucket{model}
  • swift_ttft_seconds_bucket{model}(time-to-first-token)
  • swift_tpot_seconds_bucket{model}(time-per-output-token,可选)
  1. 并发与资源(可选)
  • swift_inflight_requests{model}
  • swift_batch_size{model}
  • GPU 显存/利用率(若实现复杂可后续迭代)
配置建议

新增部署参数(示例):

  • --enable_metrics true|false(默认 false)
  • --metrics_port 9000
  • --metrics_path /metrics
  • --metrics_labels key=value,...(可选)
兼容性与实现建议
  • 默认关闭,避免影响现有行为
  • 指标端点与推理端口解耦,减少干扰
  • 使用 Prometheus 客户端统一导出(Histogram + Counter + Gauge)
验收标准
  • 开启后可通过 HTTP 抓取 /metrics
  • 压测下可看到 QPS、P50/P95/P99、tokens/s、TTFT 指标变化
  • 文档提供 Grafana Dashboard 示例(可后续补充)
Pull Request / Pull Request 信息

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names the swift deploy entry point and existing health endpoints, but no files or tests. Start by locating the deploy implementation and its health-check server, then trace request, token, queue, and latency handling. Done means an opt-in /metrics endpoint exposes the agreed metrics, configuration works as described, and the documented acceptance checks pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
prometheus, python
Domain
backend, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.