airbytehq / airbytehq/PyAirbyte

feat: Enhance MCP tools for cross-workspace connector migration

Đang mở
#978 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
344
Fork
77
Merge trung bình
1 ngày 11 giờ
Pull request đã merge (30 ngày)
35

Mô tả

## Summary

Enhance PyAirbyte MCP (Coral MCP) tools to support cross-workspace connector duplication and migration workflows (e.g., self-managed → Cloud, Cloud → Cloud). This was identified during an exploration of the end-to-end migration process using Coral MCP and Ops MCP tools.

**Requested by:** @aaronsteers

**Related Devin session:** https://app.devin.ai/sessions/eb636b8d14fc4895b3b894ea399f998e

> **Note:** Setting catalog/state artifacts is intentionally out of scope here — those write operations are handled by the Ops MCP `connection_medic` tools (behind `AIRBYTE_OPS_MEDIC_MODE`).

---

## Context: Migration Workflow

A connector migration requires these steps:
1. **Read** source connector config (type, name, credentials/config)
2. **Read** destination connector config
3. **Read** connection metadata (streams, sync modes, schedule, enabled status)
4. **Read** configured catalog (already supported via `get_connection_artifact`)
5. **Read** connection state (already supported via `get_connection_artifact`)
6. **Deploy** source to target workspace (supported via `deploy_source_to_cloud`)
7. **Deploy** destination to target workspace (supported via `deploy_destination_to_cloud`)
8. **Create** connection in target workspace (supported via `create_connection_on_cloud`)
9. **Import** catalog to target connection (handled by Ops MCP medic mode)
10. **Import** state to target connection (handled by Ops MCP medic mode)

Steps 4-5 and 6-10 are already covered. **Steps 1-3 have gaps** documented below.

---

## P0: Critical Gaps

### 1. Add source/destination config to describe responses (or new tool)

**Problem:** `describe_cloud_source` and `describe_cloud_destination` return name, ID, URL, and `connector_definition_id` — but **not the connector configuration**. Without the config, you cannot recreate the connector in a target workspace.

**Affected code:**
- `airbyte/mcp/cloud.py` — `CloudSourceDetails` model (line ~94-104) has no config field
- `airbyte/mcp/cloud.py` — `CloudDestinationDetails` model (line ~107-117) has no config field
- `airbyte/mcp/cloud.py` — `describe_cloud_source()` function (line ~921-948) strips config from response
- `airbyte/mcp/cloud.py` — `describe_cloud_destination()` function (line ~957-984) strips config from response

**Proposed fix:** Add a `configuration` field to `CloudSourceDetails` and `CloudDestinationDetails`, populated from the `SourceResponse.configuration` / `DestinationResponse.configuration` API data. The API already masks secrets, so sensitive values will appear redacted — the user would need to re-enter credentials for the target workspace.

Implementation Details

The underlying API calls (`api_util.get_source()` and `api_util.get_destination()`) already return `SourceResponse` / `DestinationResponse` objects that include `configuration`. The MCP tool just needs to pass it through. The `_connector_info` is already fetched and cached when `source.name` is accessed.

Key files:
- `airbyte/mcp/cloud.py`: `CloudSourceDetails`, `CloudDestinationDetails`, `describe_cloud_source()`, `describe_cloud_destination()`
- `airbyte/cloud/connectors.py`: `CloudSource._fetch_connector_info()` (line ~194), `CloudDestination._fetch_connector_info()` (line ~279)

### 2. Add `connector_name` to describe source/dest responses

**Problem:** `describe_cloud_source` returns `connector_definition_id` (a UUID), but `deploy_source_to_cloud` requires `source_connector_name` (e.g., `"source-postgres"`). There is no tool that maps definition_id → connector name, making it impossible to programmatically recreate a connector.

**Affected code:**
- `airbyte/mcp/cloud.py` — `CloudSourceDetails` model has `connector_definition_id` but no `connector_name`
- `airbyte/mcp/cloud.py` — `CloudDestinationDetails` model same issue

**Proposed fix:** Add `connector_name` field (e.g., `"source-postgres"`) to both response models. The `SourceResponse` object includes `source_type` which can be mapped to the connector name format.

---

## P1: Important Gaps

### 3. Add `config_api_root` parameter to Coral MCP tools

**Problem:** The Ops MCP tools (`get_connection_state`, `get_connection_catalog`) accept a `config_api_root` parameter to target self-managed Airbyte instances. But all Coral MCP tools resolve the API root from env vars only — they have no equivalent override parameter. This prevents reading from or writing to self-managed instances without switching environment variables.

**Affected code:**
- `airbyte/mcp/cloud.py` — `_get_cloud_workspace()` function (line ~238-270) hardcodes API root from env/config
- All Coral MCP cloud tools that call `_get_cloud_workspace()`

**Proposed fix:** Add an optional `config_api_root` parameter to `_get_cloud_workspace()` and propagate it to the relevant MCP tool signatures (at minimum: `describe_cloud_source`, `describe_cloud_destination`, `describe_cloud_connection`, `get_connection_artifact`, `deploy_source_to_cloud`, `deploy_destination_to_cloud`, `create_connection_on_cloud`).

Reference: Ops MCP approach

The Ops MCP already implements this pattern in `connection_state.py`:

```python
config_api_root: Annotated[
str | None,
Field(
description="Optional API root URL override. "
"Defaults to Airbyte Cloud. "
"Use this to target local or self-hosted deployments.",
default=None,
),
] = None,
```

And passes it through: `api_root=config_api_root or constants.CLOUD_API_ROOT`

### 4. Add schedule and enabled status to `describe_cloud_connection`

**Problem:** `describe_cloud_connection` returns source_id, dest_id, selected_streams, and table_prefix — but omits schedule type (cron/manual), cron expression, and enabled/disabled status. These are needed for a faithful migration.

**Affected code:**
- `airbyte/mcp/cloud.py` — `CloudConnectionDetails` model (line ~120-140) missing schedule and status fields
- `airbyte/mcp/cloud.py` — `describe_cloud_connection()` function (line ~993-1022)

**Proposed fix:** Add `schedule_type`, `cron_expression`, and `status` (active/inactive) fields to `CloudConnectionDetails`, populated from the `ConnectionResponse` data that is already fetched.

---

## Checklist

- [ ] P0: Add config to `describe_cloud_source` / `describe_cloud_destination` responses
- [ ] P0: Add `connector_name` to source/dest describe responses
- [ ] P1: Add `config_api_root` parameter to Coral MCP cloud tools
- [ ] P1: Add schedule/status to `describe_cloud_connection` response

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.