airbytehq / airbytehq/PyAirbyte

feat: Enhance MCP tools for cross-workspace connector migration

Abierto
#978 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
344
Forks
77
Merge medio
1 d 11 h
PR fusionados (30 d)
35

Descripción

## Summary

Enhance PyAirbyte MCP (Coral MCP) tools to support cross-workspace connector duplication and migration workflows (e.g., self-managed → Cloud, Cloud → Cloud). This was identified during an exploration of the end-to-end migration process using Coral MCP and Ops MCP tools.

**Requested by:** @aaronsteers

**Related Devin session:** https://app.devin.ai/sessions/eb636b8d14fc4895b3b894ea399f998e

> **Note:** Setting catalog/state artifacts is intentionally out of scope here — those write operations are handled by the Ops MCP `connection_medic` tools (behind `AIRBYTE_OPS_MEDIC_MODE`).

---

## Context: Migration Workflow

A connector migration requires these steps:
1. **Read** source connector config (type, name, credentials/config)
2. **Read** destination connector config
3. **Read** connection metadata (streams, sync modes, schedule, enabled status)
4. **Read** configured catalog (already supported via `get_connection_artifact`)
5. **Read** connection state (already supported via `get_connection_artifact`)
6. **Deploy** source to target workspace (supported via `deploy_source_to_cloud`)
7. **Deploy** destination to target workspace (supported via `deploy_destination_to_cloud`)
8. **Create** connection in target workspace (supported via `create_connection_on_cloud`)
9. **Import** catalog to target connection (handled by Ops MCP medic mode)
10. **Import** state to target connection (handled by Ops MCP medic mode)

Steps 4-5 and 6-10 are already covered. **Steps 1-3 have gaps** documented below.

---

## P0: Critical Gaps

### 1. Add source/destination config to describe responses (or new tool)

**Problem:** `describe_cloud_source` and `describe_cloud_destination` return name, ID, URL, and `connector_definition_id` — but **not the connector configuration**. Without the config, you cannot recreate the connector in a target workspace.

**Affected code:**
- `airbyte/mcp/cloud.py` — `CloudSourceDetails` model (line ~94-104) has no config field
- `airbyte/mcp/cloud.py` — `CloudDestinationDetails` model (line ~107-117) has no config field
- `airbyte/mcp/cloud.py` — `describe_cloud_source()` function (line ~921-948) strips config from response
- `airbyte/mcp/cloud.py` — `describe_cloud_destination()` function (line ~957-984) strips config from response

**Proposed fix:** Add a `configuration` field to `CloudSourceDetails` and `CloudDestinationDetails`, populated from the `SourceResponse.configuration` / `DestinationResponse.configuration` API data. The API already masks secrets, so sensitive values will appear redacted — the user would need to re-enter credentials for the target workspace.

Implementation Details

The underlying API calls (`api_util.get_source()` and `api_util.get_destination()`) already return `SourceResponse` / `DestinationResponse` objects that include `configuration`. The MCP tool just needs to pass it through. The `_connector_info` is already fetched and cached when `source.name` is accessed.

Key files:
- `airbyte/mcp/cloud.py`: `CloudSourceDetails`, `CloudDestinationDetails`, `describe_cloud_source()`, `describe_cloud_destination()`
- `airbyte/cloud/connectors.py`: `CloudSource._fetch_connector_info()` (line ~194), `CloudDestination._fetch_connector_info()` (line ~279)

### 2. Add `connector_name` to describe source/dest responses

**Problem:** `describe_cloud_source` returns `connector_definition_id` (a UUID), but `deploy_source_to_cloud` requires `source_connector_name` (e.g., `"source-postgres"`). There is no tool that maps definition_id → connector name, making it impossible to programmatically recreate a connector.

**Affected code:**
- `airbyte/mcp/cloud.py` — `CloudSourceDetails` model has `connector_definition_id` but no `connector_name`
- `airbyte/mcp/cloud.py` — `CloudDestinationDetails` model same issue

**Proposed fix:** Add `connector_name` field (e.g., `"source-postgres"`) to both response models. The `SourceResponse` object includes `source_type` which can be mapped to the connector name format.

---

## P1: Important Gaps

### 3. Add `config_api_root` parameter to Coral MCP tools

**Problem:** The Ops MCP tools (`get_connection_state`, `get_connection_catalog`) accept a `config_api_root` parameter to target self-managed Airbyte instances. But all Coral MCP tools resolve the API root from env vars only — they have no equivalent override parameter. This prevents reading from or writing to self-managed instances without switching environment variables.

**Affected code:**
- `airbyte/mcp/cloud.py` — `_get_cloud_workspace()` function (line ~238-270) hardcodes API root from env/config
- All Coral MCP cloud tools that call `_get_cloud_workspace()`

**Proposed fix:** Add an optional `config_api_root` parameter to `_get_cloud_workspace()` and propagate it to the relevant MCP tool signatures (at minimum: `describe_cloud_source`, `describe_cloud_destination`, `describe_cloud_connection`, `get_connection_artifact`, `deploy_source_to_cloud`, `deploy_destination_to_cloud`, `create_connection_on_cloud`).

Reference: Ops MCP approach

The Ops MCP already implements this pattern in `connection_state.py`:

```python
config_api_root: Annotated[
str | None,
Field(
description="Optional API root URL override. "
"Defaults to Airbyte Cloud. "
"Use this to target local or self-hosted deployments.",
default=None,
),
] = None,
```

And passes it through: `api_root=config_api_root or constants.CLOUD_API_ROOT`

### 4. Add schedule and enabled status to `describe_cloud_connection`

**Problem:** `describe_cloud_connection` returns source_id, dest_id, selected_streams, and table_prefix — but omits schedule type (cron/manual), cron expression, and enabled/disabled status. These are needed for a faithful migration.

**Affected code:**
- `airbyte/mcp/cloud.py` — `CloudConnectionDetails` model (line ~120-140) missing schedule and status fields
- `airbyte/mcp/cloud.py` — `describe_cloud_connection()` function (line ~993-1022)

**Proposed fix:** Add `schedule_type`, `cron_expression`, and `status` (active/inactive) fields to `CloudConnectionDetails`, populated from the `ConnectionResponse` data that is already fetched.

---

## Checklist

- [ ] P0: Add config to `describe_cloud_source` / `describe_cloud_destination` responses
- [ ] P0: Add `connector_name` to source/dest describe responses
- [ ] P1: Add `config_api_root` parameter to Coral MCP cloud tools
- [ ] P1: Add schedule/status to `describe_cloud_connection` response

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.