forcedotcom / forcedotcom/d360-mcp-server
GLUE_IRC file federation data streams cannot be created via Connect REST API
- Dominant language
- Java
- Stars
- 16
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
## Problem
`d360_datastream_create` cannot create GLUE_IRC (AWS Glue Iceberg REST Catalog) file federation data streams end-to-end. The underlying Connect REST API (`POST /ssot/data-streams`) has two blocking gaps.
## Gap 1: `advancedAttributes.schema` rejects all values for single-level namespaces
The API requires `advancedAttributes.schema` (maps to `externalSchemaName`), but AWS Glue Iceberg REST Catalog uses single-level namespaces (just a database name, no schema). Passing:
- `null` → NPE in the backend
- `""` (empty string) → "Invalid namespace name"
- Any string value → "Invalid namespace name" (the backend concatenates `schema.table` and tries to resolve a two-level path that doesn't exist in Glue)
The Salesforce UI bypasses this by calling an internal Aura controller (`DataStreamDeploymentController/ACTION$deploy`) that passes `externalSchemaName: null` directly, avoiding the Connect API validation entirely.
## Gap 2: No `connectorType` value maps to GLUE_IRC for data stream creation
The MCP server source shows all external connectors use `connectorType: "DataConnector"` with `connectorDetails: {"name": ""}`. There's no documented way to specify `GLUE_IRC` as a connector type through the public API — the UI uses a separate internal flow.
## Expected behavior
The Connect REST API should:
1. Allow `advancedAttributes.schema` to be optional (or accept `null`) for connectors that use single-level namespaces (GLUE_IRC, and potentially future Iceberg catalog types like S3 Tables)
2. Document how to specify GLUE_IRC file federation data streams through the public API, or expose the same `externalSchemaName: null` behavior the UI uses
## Impact
Without this, Zero Copy / BYOL data streams for AWS Glue Iceberg REST Catalog can only be created through the Salesforce UI — not via the MCP server, Connect API, or any other programmatic client.
## Related
- PR: javatestcase/d360-mcp-server#1 (fixes a separate `@NotNull` validation issue on `connectorDetails`)
- [AWS Glue Iceberg REST endpoint docs](https://docs.aws.amazon.com/glue/latest/dg/connect-glu-iceberg-rest.html) (single-level namespace only)
Contributor guide
Research direction
Start by tracing d360_datastream_create and the MCP server source to confirm the connector payload sent to POST /ssot/data-streams. Compare the public Connect REST API behavior with the Salesforce UI flow and AWS Glue Iceberg's single-level namespace requirements; done means GLUE_IRC streams can be created programmatically with documented connector and schema handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- api, backend, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100