airbytehq / airbytehq/PyAirbyte

Reevaluate MCP telemetry identity: Segment write key and user/analytics ID

未关闭
#1,129 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
344
派生
77
平均合并
1 天 11 小时
30 天内合并 PR
35

描述

## Context

[#1124](https://github.com/airbytehq/PyAirbyte/pull/1124) wires the shared `fastmcp-extensions` tool-call telemetry into PyAirbyte's MCP server, so every tool call (stdio and hosted HTTP) now emits an `mcp_tool_call` Segment event.

Two identity/attribution decisions were made expediently in that PR and deserve a real design pass before we build reporting on top of them.

## 1. Segment write key

Today the MCP tool-call stream defaults to PyAirbyte's existing app tracking key (`PYAIRBYTE_APP_TRACKING_KEY`), overridable per-deployment via the `AIRBYTE_MCP_SEGMENT_WRITE_KEY` env var, which no deployment currently sets. Consequences:

- Hosted Cloud MCP, local stdio MCP, and plain PyAirbyte library usage all share one Segment source, so they can only be separated downstream by event name and the `is_hosted_mcp` property — not by source.
- Volume/retention/sampling policy cannot be tuned for MCP traffic independently of library traffic.
- Rotating or revoking the key affects both streams.

Options to evaluate: keep the shared key; a single MCP-specific key set on all MCP deployments; or a per-deployment key (Cloud MCP support bot vs. local vs. other hosted MCP servers), which is already possible by setting `AIRBYTE_MCP_SEGMENT_WRITE_KEY` and requires no code change here.

## 2. `user_id` / analytics ID

The shared middleware sends a fixed `user_id` (`airbyte-mcp` for PyAirbyte's server) and no `anonymousId`. That is deliberately privacy-safe, but it means:

- We can count tool calls, but not unique callers, workspaces, or organizations.
- Hosted Cloud MCP serves the Airbyte Cloud support bot and many Devin sessions from one container, so even the pre-existing PyAirbyte `anonymous_id` (derived from `~/.airbyte/analytics.yml` on the local filesystem) would be per-container, not per-caller — it is not a usable substitute.
- The regular PyAirbyte `install`/`sync` streams *do* carry `anonymous_id`, so MCP events are a different shape from everything else in the same dataset.

Candidates worth weighing against privacy/PII constraints:

- Keep the fixed label (status quo): counts only.
- Local stdio: reuse the existing PyAirbyte `anonymous_id` so local MCP matches the rest of PyAirbyte telemetry.
- Hosted: derive a stable, non-PII caller identifier from the authenticated request (e.g. a hash of workspace/organization ID, or of the client ID) and send it as `anonymousId` or a property — never a raw email or user ID.
- Distinguish the calling client (support bot vs. Devin vs. human IDE) via an explicit client-supplied property rather than inferring it.

## Why it matters

[airbyte-warehouse#3472](https://github.com/airbytehq/airbyte-warehouse/pull/3472) models this stream and [internal.airbyte.ai#128](https://github.com/airbytehq/internal.airbyte.ai/pull/128) will report on it. Any "unique users of the MCP" metric is impossible until this is settled, and changing identity semantics after the fact makes historical data non-comparable — so it is cheaper to decide now, while the stream is new.

## Scope

Decision + follow-up implementation; no change required to unblock #1124.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。