modelscope / modelscope/ms-swift
[Feature] swift deploy 增加可观测性指标暴露(QPS/延时/tokens)与 Prometheus 集成
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Checklist / 检查清单
- I have searched existing issues, and this is a new feature request. / 我已经搜索过现有的 issues,确认这是一个新的 Feature Request。
Feature Request Description / Feature Request 描述
问题背景
目前 swift deploy 已支持基础健康检查(如 health/ping)以及日志侧统计(如 log_interval 打印 tokens/s),但缺少类似 vLLM server 的标准化监控指标暴露能力,不利于生产环境容量规划、SLO 监控和告警。
期望能力
希望 swift deploy 增加内置 metrics(建议 Prometheus 格式)并提供标准抓取端点,例如 /metrics。
建议包含的核心指标:
- 吞吐与请求
swift_requests_total{model,route,status}swift_request_qps(或由 total + rate 计算)swift_tokens_input_total{model}swift_tokens_output_total{model}swift_tokens_per_second{model}
- 时延
swift_request_latency_seconds_bucket{model,route}(Histogram)swift_queue_latency_seconds_bucket{model}swift_ttft_seconds_bucket{model}(time-to-first-token)swift_tpot_seconds_bucket{model}(time-per-output-token,可选)
- 并发与资源(可选)
swift_inflight_requests{model}swift_batch_size{model}- GPU 显存/利用率(若实现复杂可后续迭代)
配置建议
新增部署参数(示例):
--enable_metrics true|false(默认 false)--metrics_port 9000--metrics_path /metrics--metrics_labels key=value,...(可选)
兼容性与实现建议
- 默认关闭,避免影响现有行为
- 指标端点与推理端口解耦,减少干扰
- 使用 Prometheus 客户端统一导出(Histogram + Counter + Gauge)
验收标准
- 开启后可通过 HTTP 抓取
/metrics - 压测下可看到 QPS、P50/P95/P99、tokens/s、TTFT 指标变化
- 文档提供 Grafana Dashboard 示例(可后续补充)
Pull Request / Pull Request 信息
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names the swift deploy entry point and existing health endpoints, but no files or tests. Start by locating the deploy implementation and its health-check server, then trace request, token, queue, and latency handling. Done means an opt-in /metrics endpoint exposes the agreed metrics, configuration works as described, and the documented acceptance checks pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- prometheus, python
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100