airbytehq / airbytehq/PyAirbyte

Tool- and domain-specific telemetry for MCP server

Abierto
#884 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
344
Forks
77
Merge medio
1 d 11 h
PR fusionados (30 d)
35

Descripción

## Summary

This issue proposes enhancing the MCP server telemetry to track more granular information about MCP tool usage and domain-specific activity.

## Background

The current telemetry module (`airbyte/_util/telemetry.py`) tracks MCP mode via a boolean flag in the `flags` dictionary:

```python
def get_env_flags() -> dict[str, Any]:
flags: dict[str, bool | str] = {
"CI": meta.is_ci(),
"LANGCHAIN": meta.is_langchain(),
"MCP": meta.is_mcp_mode(), # Current MCP tracking
"NOTEBOOK_RUNTIME": ...
}
```

This allows us to filter events by MCP usage, but doesn't provide insight into which specific MCP tools are being used or how they're being used.

## Proposed Enhancements

### 1. Track MCP Tool Invocations
Add telemetry for which MCP tools are being called:
- `list_connectors`, `get_connector_info`, `get_api_docs_urls` (registry domain)
- `validate_connector_config`, `list_source_streams`, `sync_source_to_cache`, `run_sql_query` (local domain)
- `deploy_source_to_cloud`, `create_connection_on_cloud`, `run_cloud_sync` (cloud domain)

### 2. Track MCP Tool Success/Failure Rates
For each tool invocation, track:
- Tool name
- Domain (registry, local, cloud)
- Success/failure status
- Error type (if failed)

### 3. Track MCP Session Metrics
- Session duration
- Number of tool calls per session
- Tool call sequences (for understanding common workflows)

## Implementation Considerations

- Privacy: Continue to hash any potentially identifying information
- Performance: Use async/non-blocking telemetry calls
- Opt-out: Respect existing `DO_NOT_TRACK` environment variable

## Related

- PR for MCP telemetry reports in airbyte-dataops-poc-internal: https://github.com/airbytehq/airbyte-dataops-poc-internal/pull/14
- MCP server implementation: `airbyte/mcp/`
- Telemetry module: `airbyte/_util/telemetry.py`

---

*Requested by AJ Steers (@aaronsteers)*

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.