stacklok / stacklok/toolhive

Add usage metrics for vMCP product telemetry

Open
#2,851 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement vmcp
Dominant language
Go
Stars
2.2k
Forks
300
Avg merge
1d 15h
Merged PRs (30d)
184

Description

Summary

Implement product telemetry for vMCP to track usage patterns and inform product decisions.

Background

ToolHive has existing patterns for product telemetry:

  • Operator telemetry in pkg/operator/telemetry/ for instance tracking
  • Version checking and update notifications
  • ConfigMap-based instance ID persistence

vMCP needs similar product telemetry to understand:

  • How many vMCP instances are deployed
  • What configurations are commonly used
  • Tool call patterns and volumes

Reusable Patterns

Pattern Location Description
Instance Tracking pkg/operator/telemetry/telemetry.go Instance ID generation
Update Checking pkg/updates/ Periodic telemetry collection

Implementation

Metrics to Track

Deployment Metrics:

  • Number of vMCP instances
  • Backends per vMCP (count distribution)
  • Conflict resolution strategies used (prefix/priority/manual)
  • Auth types configured (OIDC, anonymous, token exchange)

Usage Metrics:

  • Tool calls per vMCP instance (aggregated)
  • Composite tools defined and executed
  • Backend distribution (which backends are most used)

Operational Metrics:

  • vMCP uptime
  • Configuration changes over time
Implementation Steps
  1. Define telemetry data model

    • What data points to collect
    • Privacy considerations (no PII, no tool arguments)
  2. Add collection mechanism

    • Periodic aggregation of usage data
    • Instance identification (anonymous ID)
  3. Add reporting endpoint

    • Optional opt-in telemetry reporting
    • Clear documentation of what's collected

Privacy Considerations

  • No personally identifiable information (PII)
  • No tool arguments or resource content
  • No authentication tokens or secrets
  • Aggregated counts only (not individual requests)
  • Clear opt-out mechanism

Acceptance Criteria

  • Usage metrics defined and documented
  • Collection mechanism implemented
  • Privacy-preserving (no PII, aggregated data only)
  • Opt-out mechanism available
  • Documentation of what's collected
  • Kubernetes and CLI deployment support

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing patterns in pkg/operator/telemetry/telemetry.go and pkg/updates/. Define the telemetry data model and privacy boundaries before designing collection and reporting for both Kubernetes and CLI deployments. Done means documented aggregated metrics, opt-out support, privacy safeguards, and implementation coverage for the listed acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
backend, cloud, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.