airbytehq / airbytehq/PyAirbyte

Reevaluate MCP telemetry identity: Segment write key and user/analytics ID

Abierto
#1,129 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
344
Forks
77
Merge medio
1 d 11 h
PR fusionados (30 d)
35

Descripción

## Context

[#1124](https://github.com/airbytehq/PyAirbyte/pull/1124) wires the shared `fastmcp-extensions` tool-call telemetry into PyAirbyte's MCP server, so every tool call (stdio and hosted HTTP) now emits an `mcp_tool_call` Segment event.

Two identity/attribution decisions were made expediently in that PR and deserve a real design pass before we build reporting on top of them.

## 1. Segment write key

Today the MCP tool-call stream defaults to PyAirbyte's existing app tracking key (`PYAIRBYTE_APP_TRACKING_KEY`), overridable per-deployment via the `AIRBYTE_MCP_SEGMENT_WRITE_KEY` env var, which no deployment currently sets. Consequences:

- Hosted Cloud MCP, local stdio MCP, and plain PyAirbyte library usage all share one Segment source, so they can only be separated downstream by event name and the `is_hosted_mcp` property — not by source.
- Volume/retention/sampling policy cannot be tuned for MCP traffic independently of library traffic.
- Rotating or revoking the key affects both streams.

Options to evaluate: keep the shared key; a single MCP-specific key set on all MCP deployments; or a per-deployment key (Cloud MCP support bot vs. local vs. other hosted MCP servers), which is already possible by setting `AIRBYTE_MCP_SEGMENT_WRITE_KEY` and requires no code change here.

## 2. `user_id` / analytics ID

The shared middleware sends a fixed `user_id` (`airbyte-mcp` for PyAirbyte's server) and no `anonymousId`. That is deliberately privacy-safe, but it means:

- We can count tool calls, but not unique callers, workspaces, or organizations.
- Hosted Cloud MCP serves the Airbyte Cloud support bot and many Devin sessions from one container, so even the pre-existing PyAirbyte `anonymous_id` (derived from `~/.airbyte/analytics.yml` on the local filesystem) would be per-container, not per-caller — it is not a usable substitute.
- The regular PyAirbyte `install`/`sync` streams *do* carry `anonymous_id`, so MCP events are a different shape from everything else in the same dataset.

Candidates worth weighing against privacy/PII constraints:

- Keep the fixed label (status quo): counts only.
- Local stdio: reuse the existing PyAirbyte `anonymous_id` so local MCP matches the rest of PyAirbyte telemetry.
- Hosted: derive a stable, non-PII caller identifier from the authenticated request (e.g. a hash of workspace/organization ID, or of the client ID) and send it as `anonymousId` or a property — never a raw email or user ID.
- Distinguish the calling client (support bot vs. Devin vs. human IDE) via an explicit client-supplied property rather than inferring it.

## Why it matters

[airbyte-warehouse#3472](https://github.com/airbytehq/airbyte-warehouse/pull/3472) models this stream and [internal.airbyte.ai#128](https://github.com/airbytehq/internal.airbyte.ai/pull/128) will report on it. Any "unique users of the MCP" metric is impossible until this is settled, and changing identity semantics after the fact makes historical data non-comparable — so it is cheaper to decide now, while the stream is new.

## Scope

Decision + follow-up implementation; no change required to unblock #1124.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.