apache / apache/amoro

[Feature]: Build an Amoro MCP server

Open
#4,266 0 comments 0 reactions 0 assignees View on GitHub
type:feature
Dominant language
Java
Stars
1.2k
Forks
395
Avg merge
4d 10h
Merged PRs (30d)
33

Description

## Description

Introduce an official, standalone MCP Server for Apache Amoro. The server will provide stable, discoverable, diagnostic-oriented tools for MCP clients and AI agents while using the existing AMS REST GET APIs.

This issue tracks [AIP-6: Introduce an MCP Server for Apache Amoro](https://cwiki.apache.org/confluence/spaces/AMORO/pages/440304643/AIP-6+Introduce+an+MCP+Server+for+Apache+Amoro). The proposal is based on the current `master` branch.

## Use case/motivation

MCP clients and AI agents currently need detailed knowledge of AMS REST routes, authentication, request signing, pagination conventions, and response structures to inspect Amoro catalogs, tables, snapshots, and self-optimizing status.

An official MCP Server would provide a stable semantic interface for catalog discovery, table inspection, bounded snapshot queries, and self-optimizing diagnostics without exposing the complete AMS REST API to models.

## Goals

- Provide a read-only MCP Server for discovery and diagnosis.
- Keep the server independently deployable and maintained in the Amoro monorepo.
- Reuse existing AMS REST GET APIs without changing AMS storage, metadata, or public REST contracts.
- Apply strict validation, response projection, redaction, pagination, and size limits.

## Non-goals

- Creating tables, changing configurations, cancelling optimization, scaling resources, or performing any other write operation.
- Providing a generic REST proxy, arbitrary URLs, arbitrary HTTP methods, or raw AMS response passthrough.
- Exposing file-level lists, operation history, or enterprise-specific authentication mechanisms.
- Publishing a PyPI package in the first release.

## Describe the solution

Add an independent Python/FastMCP service that communicates with AMS through an explicit allowlist of REST GET actions.

The public HTTP endpoints will be:

- `/mcp`: stateless Streamable HTTP MCP endpoint.
- `/health/live`: process liveness endpoint.
- `/health/ready`: bounded AMS readiness check.

The initial MCP tool contract contains 11 tools:

- `list_catalogs`
- `list_databases`
- `list_tables`
- `get_table_details`
- `list_table_snapshots`
- `list_optimizing_processes`
- `list_optimizing_tasks`
- `list_optimizers`
- `list_optimizing_tables`
- `list_resource_groups`
- `check_ams_connectivity`

`list_table_snapshots` will return only paginated, projected, and redacted commit and summary metrics. File details and operation history remain out of scope.

Each request will follow this fixed flow:

1. Strict parameter validation.
2. Resolution to an allowlisted GET action and encoded path.
3. A bounded request to AMS.
4. Field projection and sensitive-data redaction.
5. Response-size enforcement.
6. A structured MCP result.

The public configuration will cover the listen address, AMS address, authentication mode, TLS, timeouts, retries, Host and Origin allowlists, and response-size limits.

For local deployments or trusted gateways, the server can run without end-user authentication. It can also accept Amoro OpenAPI credentials per request and forward them only for that request. Server-to-AMS authentication can use no credentials, Basic authentication, or a signed token. Credentials must not be included in results, logs, authentication caches, or exception details.

The source, lock file, tests, Dockerfile, and independent CI checks will be maintained in the Amoro monorepo. The proposed container image is `apache/amoro-mcp`; the first release does not require a PyPI package.

## Compatibility and migration

This is an additive feature. It does not change the AMS database, storage formats, or existing REST APIs. Users who do not deploy the MCP Server will see no behavior change and require no migration.

## Test plan

The merge gates will cover:

- tool registration, parameter validation, path encoding, pagination, the GET allowlist, and response redaction;
- concurrent request credential isolation, authentication failures, TLS, log redaction, and response-size limits;
- end-to-end catalog, table, snapshot, and self-optimizing diagnostic flows against a real AMS instance;
- `pytest`, Ruff lint and format checks, mypy, lock-file checks, Apache RAT, dependency-license checks, and a Docker build.

A non-binding reference implementation currently passes 33 tests, Ruff lint, mypy, and the lock-file check. Four files still fail the Ruff format check, so its CI should not yet be considered fully green.

## Related issues and documents

- AIP-6: https://cwiki.apache.org/confluence/spaces/AMORO/pages/440304643/AIP-6+Introduce+an+MCP+Server+for+Apache+Amoro
- Current master baseline: https://github.com/apache/amoro/commit/27ae8d55f2d300acb0d123602d90928ab7349b6b

## Are you willing to submit a PR?

- [x] Yes, I am willing to submit a PR!

## Code of Conduct

- [x] I agree to follow this project's Code of Conduct.

Contributor guide

Open the contributing guide

Research direction

Start by reading AIP-6 and reviewing the reference implementation described in the issue, then run its pytest, Ruff, mypy, and lock-file checks. Confirm the 11 read-only tools, health endpoints, validation, redaction, pagination, authentication isolation, and response limits against the stated test plan; done includes passing CI checks and a Docker build.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, devops, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.