stacklok / stacklok/toolhive

Deprecate redundant custom duration metrics in favor of OTEL MCP standard metrics

Open
#3,791 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement go telemetry
Dominant language
Go
Stars
2.2k
Forks
300
Avg merge
1d 15h
Merged PRs (30d)
184

Description

Summary

ToolHive now emits OTEL MCP semantic convention duration metrics alongside the original custom metrics. Two custom histogram metrics are redundant and should be deprecated and eventually removed.

Redundant Metrics

Custom Metric (redundant) Standard Replacement Location
toolhive_mcp_request_duration mcp.server.operation.duration pkg/telemetry/middleware.go
toolhive_vmcp_backend_requests_duration mcp.client.operation.duration pkg/vmcp/server/telemetry.go

Both pairs measure the same duration in seconds using the same histogram bucket boundaries ([0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10, 30, 60, 120, 300]). The standard metrics use richer OTEL MCP semconv attributes (mcp.method.name, network.transport, error.type, gen_ai.tool.name, etc.).

Proposed Deprecation Plan

  1. Current release: Both custom and standard metrics emitted (status quo)
  2. Next minor release: Log a deprecation warning at startup when the custom metrics are being consumed (if detectable), and document the deprecation in release notes
  3. Following minor release: Remove the custom metrics entirely

Non-Redundant Metrics (No Action Needed)

These custom metrics have no direct OTEL MCP standard equivalent and should be retained:

  • toolhive_mcp_requests (Counter) — no standard counter exists
  • toolhive_mcp_tool_calls (Counter) — scoped convenience counter for tools/call
  • toolhive_mcp_active_connections (UpDownCounter) — no standard gauge equivalent
  • toolhive_vmcp_backend_requests (Counter) — no standard counter exists
  • toolhive_vmcp_backend_errors (Counter) — no standard error counter
  • toolhive_vmcp_backends_discovered (Gauge) — ToolHive-specific operational metric
  • toolhive_vmcp_workflow_* (3 metrics) — composite workflows are a ToolHive concept

Related

  • Parent issue: #3399 (OTEL telemetry improvements)
  • #3764 (implement mcp.server.session.duration)
  • #3765 (JSON-RPC error code capture in error.type)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading pkg/telemetry/middleware.go and pkg/vmcp/server/telemetry.go to compare the two custom duration metrics with their OTEL MCP replacements and confirm their consumers. Check the release-note process and determine how deprecation warnings can be detected; done means the agreed staged deprecation is documented and implemented without changing the non-redundant metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.