forcedotcom / forcedotcom/d360-mcp-server

GLUE_IRC file federation data streams cannot be created via Connect REST API

Open
#15 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
16
Forks
12
PR merge metrics
No merged PRs in 30d

Description

## Problem

`d360_datastream_create` cannot create GLUE_IRC (AWS Glue Iceberg REST Catalog) file federation data streams end-to-end. The underlying Connect REST API (`POST /ssot/data-streams`) has two blocking gaps.

## Gap 1: `advancedAttributes.schema` rejects all values for single-level namespaces

The API requires `advancedAttributes.schema` (maps to `externalSchemaName`), but AWS Glue Iceberg REST Catalog uses single-level namespaces (just a database name, no schema). Passing:
- `null` → NPE in the backend
- `""` (empty string) → "Invalid namespace name"
- Any string value → "Invalid namespace name" (the backend concatenates `schema.table` and tries to resolve a two-level path that doesn't exist in Glue)

The Salesforce UI bypasses this by calling an internal Aura controller (`DataStreamDeploymentController/ACTION$deploy`) that passes `externalSchemaName: null` directly, avoiding the Connect API validation entirely.

## Gap 2: No `connectorType` value maps to GLUE_IRC for data stream creation

The MCP server source shows all external connectors use `connectorType: "DataConnector"` with `connectorDetails: {"name": ""}`. There's no documented way to specify `GLUE_IRC` as a connector type through the public API — the UI uses a separate internal flow.

## Expected behavior

The Connect REST API should:
1. Allow `advancedAttributes.schema` to be optional (or accept `null`) for connectors that use single-level namespaces (GLUE_IRC, and potentially future Iceberg catalog types like S3 Tables)
2. Document how to specify GLUE_IRC file federation data streams through the public API, or expose the same `externalSchemaName: null` behavior the UI uses

## Impact

Without this, Zero Copy / BYOL data streams for AWS Glue Iceberg REST Catalog can only be created through the Salesforce UI — not via the MCP server, Connect API, or any other programmatic client.

## Related

- PR: javatestcase/d360-mcp-server#1 (fixes a separate `@NotNull` validation issue on `connectorDetails`)
- [AWS Glue Iceberg REST endpoint docs](https://docs.aws.amazon.com/glue/latest/dg/connect-glu-iceberg-rest.html) (single-level namespace only)

Contributor guide

Open the contributing guide

Research direction

Start by tracing d360_datastream_create and the MCP server source to confirm the connector payload sent to POST /ssot/data-streams. Compare the public Connect REST API behavior with the Salesforce UI flow and AWS Glue Iceberg's single-level namespace requirements; done means GLUE_IRC streams can be created programmatically with documented connector and schema handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
api, backend, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.