github / github/copilot-cli

Cloud agent: MCP registry policy fetch fails with 401/403 on GHEC data residency, silently blocking all user-configured MCP servers

Open
#4,378 0 comments 0 reactions 0 assignees View on GitHub
triage
Dominant language
Shell
Stars
11.2k
Forks
1.9k
Avg merge
14h 16m
Merged PRs (30d)
6

Description

### Describe the bug

## Summary

On a GitHub Enterprise Cloud **with data residency** instance (`.ghe.com`), every user-configured MCP server is silently dropped from Copilot **cloud agent** sessions. Only the platform defaults (`github-mcp-server`, `playwright`) reach the model. The cause is the registry-policy fetch failing with **401/403** and falling into the fail-closed path — the same defect as #2498, where only the **404** leg was fixed.

This is the same fail-closed family as #2498, #2552, #2567, #4346 and #4349, but reached through the cloud agent, where nothing is surfaced to the user at all.

## Environment

- Surface: Copilot cloud agent (not the local CLI, not an IDE)
- Host: GitHub Enterprise Cloud with data residency, `.ghe.com`
- Two MCP servers configured in repo Settings → Copilot → MCP servers: one remote `http`, one `local` (stdio). Both valid per the documented schema, both with explicit `tools` lists.
- Enterprise MCP registry: none configured. `GET /copilot/mcp_registry` (user PAT) returns `{"mcp_registries":[{"url":"","registry_access":"allow_all",...}]}`
- `GET /copilot_internal/user` reports `is_mcp_enabled: true`, `copilot_plan: "enterprise"`

## What happens

From the CLI's own log inside the session (`~/.copilot/logs/process-*.log`):

```
[ERROR] Failed to validate SDK token (401): GitHub returned: Bad credentials
[WARNING] session.gitHubAuth.setCredentials: failed to re-resolve copilotUser for token
credential; token swap applied, plan/quota/billing metadata degraded:
Failed to fetch Copilot user info: 401 Unauthorized: {"message":"Bad credentials"}
[WARNING] Failed to fetch MCP registry policy: 401 Unauthorized.
Non-default MCP servers will be blocked until the policy can be fetched.
[ERROR] MCP server "" filtered: Could not verify server against any configured registry
[ERROR] MCP server "" filtered: Could not verify server against any configured registry
[ERROR] Using deferred connection for default Playwright server
```

Probing the endpoint from inside the session with **every** credential present (`GITHUB_COPILOT_API_TOKEN`, `GITHUB_TOKEN`, `COPILOT_SDK_AUTH_TOKEN`, …): **403** on `api..ghe.com/copilot/mcp_registry`, **401** on `api.github.com/copilot/mcp_registry`. The same endpoint returns **200 `allow_all`** to a normal user PAT.

## Why this is a bug, not policy

1. **404 is tolerated, 401/403 is not.** The client special-cases a 404 as "no registry configured, allow all", but any other failure throws into the fail-closed catch that blocks every server without `isDefaultServer`. A tenant with *no registry configured* is therefore treated as if it prohibited everything, purely because its endpoint answers 403 instead of 404. #2498 fixed the 404 leg; the auth-failure leg has the identical consequence.
2. **This filter is documented as not applying to the cloud agent.** Per the MCP-management support matrix, "Registry display" and "Allowlist enforcement" are *Not supported* for Copilot cloud agent, and the enterprise docs state private MCP registries "apply to Copilot CLI and IDEs, but not to cloud agents that run on GitHub." It is nonetheless the filter rejecting these servers.
3. **The root credential failure looks host-related.** `Bad credentials` is what `api.github.com` returns for a token minted by another host, which suggests the SDK-token validation and/or the policy fetch resolves to public github.com rather than the data-residency tenant.

## Everything upstream is healthy

Verified in-session, so the failure is purely in the policy filter:

- `GET /repos/{owner}/{repo}/copilot/cloud-agent/configuration` returns both servers as stored.
- The `Start MCP Servers` step succeeds; the local MCP proxy serves all servers' tools (`GET 127.0.0.1:2301/tools` — 62 tools, including all of both user servers').
- The stdio server's `initialize`/`tools/list` handshake works when driven by hand on the runner.
- `COPILOT_MCP_ENABLED=true`, `GITHUB_COPILOT_3P_MCP_ENABLED=true`, `COPILOT_PERMISSION_MODE=allow_all`.

## Impact

**Nothing is surfaced.** No failed step, no warning in the pull request, no line in the session transcript. The messages above exist only in the CLI's private log inside the ephemeral runner. In our case a Microsoft Learn MCP server had been silently dead in every cloud-agent session for an unknown period, and we only found it while investigating a second server. Unlike the local CLI — where `--log-level debug` reveals it — cloud-agent users have no way to observe this.

Reproduced across three different agent runtime builds.

## Suggested fix

Either (or both):

1. Treat an **auth failure** (401/403) on the registry-policy fetch the same as 404 when no registry is configured for the tenant — i.e. extend #2498's fix beyond 404.
2. Fix host/token resolution so the policy endpoint authenticates on data-residency instances.

Additionally: surface a blocked-server warning on the pull request, the way firewall blocks already are. A silent drop is indistinguishable from the agent choosing not to use a tool.

## Ruled out

`MCP servers in Copilot` policy (enabled), registry allowlist (`allow_all`), custom agent `tools:` filter (none present), server transport (remote HTTP and local stdio fail identically), config schema (validated by the Settings UI), and any client-side override (no env var, CLI flag, or settings key reaches the gate; `isDefaultServer` is rejected by the Settings schema validator).

### Affected version

_No response_

### Steps to reproduce the behavior

_No response_

### Expected behavior

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing the MCP registry-policy fetch and fail-closed filtering behind the `GET /copilot/mcp_registry` calls, using the CLI process logs and cloud-agent configuration endpoint as entry points. Compare 401/403 handling with the existing 404 behavior and verify credential host resolution; done means configured servers are not silently dropped on data-residency tenants and blocked servers produce a visible warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
api, authentication, cli, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.