NVIDIA / NVIDIA/NemoClaw

Slack socket mode can never start on v0.0.114: credential-binding denies every request (credential_unavailable) even on a fresh onboard — the credential saves where NemoClaw looks but never materializes where openshell's boundary engine looks

Open
#10,602 4 comments 0 reactions 0 assignees View on GitHub
area: sandbox area: security integration: slack
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## TL;DR for triage

On a **fresh v0.0.114 onboard** (no migration, no prior state):

```
nemoclaw onboard --name ctl-gw --agent openclaw # completes, sandbox healthy
nemoclaw ctl-gw channels add slack # interactive, real tokens, completes
```

every worker attempt then logs (sandbox container OCSF stream):

```
NET:OPEN ALLOWED node -> slack.com:443 [policy:slack engine:opa]
HTTP:POST ALLOWED POST /api/apps.connections.open [policy:slack engine:l7]
HTTP:UNKNOWN DENIED [policy:slack engine:credential-binding] [reason:credential_unavailable]
[slack] socket mode failed to start; retry N/∞ ... statusCode = 500; slack_webapi_http_error
```

Slack never receives one authenticated request; the operator only sees `doctor`'s "channels paused" hint. Reproduced identically on a bare-metal host and a QEMU/KVM VM, with two different Slack apps and freshly issued tokens, on three sandbox lineages (fresh v0.0.114 onboard; v0.0.90-created sandbox migrated per #9215's workaround, twice). The same production tokens ran for months on v0.0.90/openshell 0.0.85 — the regression window is the binding architecture itself, not a recent patch, so bisecting 0.0.109→0.0.114 likely won't find it.

## Where the chain breaks, checked link by link

Everything on the **save side** verifiably works; the **resolve side** never sees the material. Each check below is copy-pasteable against a broken deployment:

| layer | check | observed |
|---|---|---|
| credential save | `openshell provider get -slack-bridge` | credential key `SLACK_BOT_TOKEN` present; **resource version bumps (1→2) when `channels add` saves** |
| sandbox attachment | `openshell sandbox provider list ` | both slack providers listed as attached |
| config | `channels.slack.accounts.default` in openclaw.json | normalized `xoxb-/xapp-OPENSHELL-RESOLVE-ENV-…` aliases (per `src/lib/messaging/channels/slack/manifest.ts`), channel + plugin `enabled: true` |
| policy | `openshell policy get --full` | slack preset applied; opa + l7 ALLOW (log above); **no provider-composed credential entries in any revision** — the slack network-policy blocks of "worked-looking" and broken revisions are byte-identical, so binding material is not carried in the policy document |
| raw egress | `docker exec curl -sX POST https://slack.com/api/api.test` | `200 {"ok":true}` — network is fine, only injection fails |
| NemoClaw's own view | `channels add slack` on a second sandbox, same tokens | conflict guard correctly reports the first sandbox "uses the same slack credential" — **NemoClaw's save surface sees the credential the binding engine cannot resolve** |

Not fixed by any of: plain rebuild; rebuild with `SLACK_*` in the rebuild process env; managed `stop`/`start`; `gateway restart`; `channels remove`+`add` (interactive and non-interactive); `policy remove/add slack`; `openshell provider update --credential` re-push (version bumps, denial unchanged).

## Suspected mechanism (from your own source, v0.0.114 tag)

The architecture comment in `src/lib/onboard/providers.ts` (`upsertMessagingProviders`) describes the intended chain:

```
ensureMessagingBridgeProfiles <- import the profile (must exist before `provider create`)
upsertProvider(... slack -> --type nemoclaw-mcp-v1) bridge created with a sentinel token
configureMessagingBridgeRefreshes <- refresh mints the real token, overwriting the sentinel
```

and `nemoclaw-blueprint/provider-profiles/nemoclaw-mcp-v1.yaml` describes the profile as "Endpointless profile for credentials bound by **generated MCP policies**."

Observed against that design:

1. **Every rebuild** prints, up to three times: `error … profile=nemoclaw-mcp-v1 field=id custom provider profile 'nemoclaw-mcp-v1' already exists` / `Error: × provider profile import failed`. `ensureMessagingBridgeProfiles` (`src/lib/onboard/messaging-bridge-provider.ts`) probes with `provider profile export` before importing and is idempotent when the probe succeeds — but during a rebuild the gateway is restarting, the probe fails, the blind import hits `already exists`, and the run takes the failure path. The registered profile is byte-equivalent to the checked-in blueprint (only server-added `resource_version/source/scope` differ), so this is the probe racing the gateway lifecycle, not real drift.
2. `openshell provider refresh status -slack-bridge` → **"No refresh configurations found"** — the mint-the-real-token step never configures. (Slack has no co-located `provider-profile/.yaml`, so `configureMessagingBridgeRefreshes` self-gates for it — we could not determine from the NemoClaw side what is *supposed* to deliver slack's binding material to the enforcement point; whatever it is, it observably never runs.)
3. `/run/nemoclaw/` inside the sandbox is empty (no `managed-startup-runtime.env`), and the in-container gateway process env contains no `SLACK_*`/`v_*` variables — on 0.0.90/openshell 0.0.85 the process-env path is how channel tokens reached the worker.

If the generated binding material for `nemoclaw-mcp-v1`-typed providers is produced in the same phase the profile-import failure aborts, that would explain a universal failure. We can't verify past the NemoClaw/openshell boundary — the denial is emitted by openshell 0.0.106's `engine:credential-binding`, which is where a maintainer with openshell source can pick up the trail.

## Two secondary defects found while isolating this (say the word and we'll file separately)

- **Non-idempotent profile import under the rebuild race** (above) — even if benign for binding, it prints a scary triple error on every rebuild.
- **Non-interactive `channels add` silently skips the credential save**: with `NEMOCLAW_NON_INTERACTIVE=1` and tokens in `SLACK_BOT_TOKEN`/`SLACK_APP_TOKEN` env, the flow completes "successfully" but the provider resource version never bumps; only the interactive prompt path saves. Cost us most of a day of misdiagnosis.

## Environment

- NemoClaw v0.0.114 (built from tag), openshell 0.0.106 (auto-installed by rebuild preflight)
- Ubuntu 24.04; Docker 29.1.3 (bare metal) and QEMU/KVM (VM); agent OpenClaw v2026.7.1; provider ollama-local / qwen3:8b
- Both hosts keep snapshots — a full retest against a branch build takes minutes, and we're happy to run any diagnostic you want against the broken state.

Contributor guide

Open the contributing guide

Research direction

Start with src/lib/onboard/providers.ts, src/lib/onboard/messaging-bridge-provider.ts, and nemoclaw-blueprint/provider-profiles/nemoclaw-mcp-v1.yaml. Reproduce the failure and inspect openshell provider refresh status, profile-import output, and credential-binding logs across a fresh onboard. Done means Slack socket mode authenticates successfully and the credential-binding denial no longer occurs.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
infrastructure, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.