airbytehq / airbytehq/PyAirbyte
Tool- and domain-specific telemetry for MCP server
- Dominant language
- Python
- Stars
- 344
- Forks
- 77
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 35
Description
## Summary
This issue proposes enhancing the MCP server telemetry to track more granular information about MCP tool usage and domain-specific activity.
## Background
The current telemetry module (`airbyte/_util/telemetry.py`) tracks MCP mode via a boolean flag in the `flags` dictionary:
```python
def get_env_flags() -> dict[str, Any]:
flags: dict[str, bool | str] = {
"CI": meta.is_ci(),
"LANGCHAIN": meta.is_langchain(),
"MCP": meta.is_mcp_mode(), # Current MCP tracking
"NOTEBOOK_RUNTIME": ...
}
```
This allows us to filter events by MCP usage, but doesn't provide insight into which specific MCP tools are being used or how they're being used.
## Proposed Enhancements
### 1. Track MCP Tool Invocations
Add telemetry for which MCP tools are being called:
- `list_connectors`, `get_connector_info`, `get_api_docs_urls` (registry domain)
- `validate_connector_config`, `list_source_streams`, `sync_source_to_cache`, `run_sql_query` (local domain)
- `deploy_source_to_cloud`, `create_connection_on_cloud`, `run_cloud_sync` (cloud domain)
### 2. Track MCP Tool Success/Failure Rates
For each tool invocation, track:
- Tool name
- Domain (registry, local, cloud)
- Success/failure status
- Error type (if failed)
### 3. Track MCP Session Metrics
- Session duration
- Number of tool calls per session
- Tool call sequences (for understanding common workflows)
## Implementation Considerations
- Privacy: Continue to hash any potentially identifying information
- Performance: Use async/non-blocking telemetry calls
- Opt-out: Respect existing `DO_NOT_TRACK` environment variable
## Related
- PR for MCP telemetry reports in airbyte-dataops-poc-internal: https://github.com/airbytehq/airbyte-dataops-poc-internal/pull/14
- MCP server implementation: `airbyte/mcp/`
- Telemetry module: `airbyte/_util/telemetry.py`
---
*Requested by AJ Steers (@aaronsteers)*
Contributor guide
Assessment
This issue has not been assessed yet.