airbytehq / airbytehq/PyAirbyte

feat: Enhance MCP tools for cross-workspace connector migration

Offen
#978 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
344
Forks
77
Ø Merge
1 T. 11 Std.
Gemergte PRs (30 T.)
35

Beschreibung

## Summary

Enhance PyAirbyte MCP (Coral MCP) tools to support cross-workspace connector duplication and migration workflows (e.g., self-managed → Cloud, Cloud → Cloud). This was identified during an exploration of the end-to-end migration process using Coral MCP and Ops MCP tools.

**Requested by:** @aaronsteers

**Related Devin session:** https://app.devin.ai/sessions/eb636b8d14fc4895b3b894ea399f998e

> **Note:** Setting catalog/state artifacts is intentionally out of scope here — those write operations are handled by the Ops MCP `connection_medic` tools (behind `AIRBYTE_OPS_MEDIC_MODE`).

---

## Context: Migration Workflow

A connector migration requires these steps:
1. **Read** source connector config (type, name, credentials/config)
2. **Read** destination connector config
3. **Read** connection metadata (streams, sync modes, schedule, enabled status)
4. **Read** configured catalog (already supported via `get_connection_artifact`)
5. **Read** connection state (already supported via `get_connection_artifact`)
6. **Deploy** source to target workspace (supported via `deploy_source_to_cloud`)
7. **Deploy** destination to target workspace (supported via `deploy_destination_to_cloud`)
8. **Create** connection in target workspace (supported via `create_connection_on_cloud`)
9. **Import** catalog to target connection (handled by Ops MCP medic mode)
10. **Import** state to target connection (handled by Ops MCP medic mode)

Steps 4-5 and 6-10 are already covered. **Steps 1-3 have gaps** documented below.

---

## P0: Critical Gaps

### 1. Add source/destination config to describe responses (or new tool)

**Problem:** `describe_cloud_source` and `describe_cloud_destination` return name, ID, URL, and `connector_definition_id` — but **not the connector configuration**. Without the config, you cannot recreate the connector in a target workspace.

**Affected code:**
- `airbyte/mcp/cloud.py` — `CloudSourceDetails` model (line ~94-104) has no config field
- `airbyte/mcp/cloud.py` — `CloudDestinationDetails` model (line ~107-117) has no config field
- `airbyte/mcp/cloud.py` — `describe_cloud_source()` function (line ~921-948) strips config from response
- `airbyte/mcp/cloud.py` — `describe_cloud_destination()` function (line ~957-984) strips config from response

**Proposed fix:** Add a `configuration` field to `CloudSourceDetails` and `CloudDestinationDetails`, populated from the `SourceResponse.configuration` / `DestinationResponse.configuration` API data. The API already masks secrets, so sensitive values will appear redacted — the user would need to re-enter credentials for the target workspace.

Implementation Details

The underlying API calls (`api_util.get_source()` and `api_util.get_destination()`) already return `SourceResponse` / `DestinationResponse` objects that include `configuration`. The MCP tool just needs to pass it through. The `_connector_info` is already fetched and cached when `source.name` is accessed.

Key files:
- `airbyte/mcp/cloud.py`: `CloudSourceDetails`, `CloudDestinationDetails`, `describe_cloud_source()`, `describe_cloud_destination()`
- `airbyte/cloud/connectors.py`: `CloudSource._fetch_connector_info()` (line ~194), `CloudDestination._fetch_connector_info()` (line ~279)

### 2. Add `connector_name` to describe source/dest responses

**Problem:** `describe_cloud_source` returns `connector_definition_id` (a UUID), but `deploy_source_to_cloud` requires `source_connector_name` (e.g., `"source-postgres"`). There is no tool that maps definition_id → connector name, making it impossible to programmatically recreate a connector.

**Affected code:**
- `airbyte/mcp/cloud.py` — `CloudSourceDetails` model has `connector_definition_id` but no `connector_name`
- `airbyte/mcp/cloud.py` — `CloudDestinationDetails` model same issue

**Proposed fix:** Add `connector_name` field (e.g., `"source-postgres"`) to both response models. The `SourceResponse` object includes `source_type` which can be mapped to the connector name format.

---

## P1: Important Gaps

### 3. Add `config_api_root` parameter to Coral MCP tools

**Problem:** The Ops MCP tools (`get_connection_state`, `get_connection_catalog`) accept a `config_api_root` parameter to target self-managed Airbyte instances. But all Coral MCP tools resolve the API root from env vars only — they have no equivalent override parameter. This prevents reading from or writing to self-managed instances without switching environment variables.

**Affected code:**
- `airbyte/mcp/cloud.py` — `_get_cloud_workspace()` function (line ~238-270) hardcodes API root from env/config
- All Coral MCP cloud tools that call `_get_cloud_workspace()`

**Proposed fix:** Add an optional `config_api_root` parameter to `_get_cloud_workspace()` and propagate it to the relevant MCP tool signatures (at minimum: `describe_cloud_source`, `describe_cloud_destination`, `describe_cloud_connection`, `get_connection_artifact`, `deploy_source_to_cloud`, `deploy_destination_to_cloud`, `create_connection_on_cloud`).

Reference: Ops MCP approach

The Ops MCP already implements this pattern in `connection_state.py`:

```python
config_api_root: Annotated[
str | None,
Field(
description="Optional API root URL override. "
"Defaults to Airbyte Cloud. "
"Use this to target local or self-hosted deployments.",
default=None,
),
] = None,
```

And passes it through: `api_root=config_api_root or constants.CLOUD_API_ROOT`

### 4. Add schedule and enabled status to `describe_cloud_connection`

**Problem:** `describe_cloud_connection` returns source_id, dest_id, selected_streams, and table_prefix — but omits schedule type (cron/manual), cron expression, and enabled/disabled status. These are needed for a faithful migration.

**Affected code:**
- `airbyte/mcp/cloud.py` — `CloudConnectionDetails` model (line ~120-140) missing schedule and status fields
- `airbyte/mcp/cloud.py` — `describe_cloud_connection()` function (line ~993-1022)

**Proposed fix:** Add `schedule_type`, `cron_expression`, and `status` (active/inactive) fields to `CloudConnectionDetails`, populated from the `ConnectionResponse` data that is already fetched.

---

## Checklist

- [ ] P0: Add config to `describe_cloud_source` / `describe_cloud_destination` responses
- [ ] P0: Add `connector_name` to source/dest describe responses
- [ ] P1: Add `config_api_root` parameter to Coral MCP cloud tools
- [ ] P1: Add schedule/status to `describe_cloud_connection` response

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.