NVIDIA / NVIDIA/NemoClaw

Canonical provider credential resolution via separate --provider/-m flags sends mask_secret() output as literal API key (401 Invalid API key)

Open
#7,361 15 comments 0 reactions 0 assignees View on GitHub
area: cli area: providers integration: hermes
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

### Investigation Summary

- `hermes -z ... --provider opencode-zen -m ` fails with a generic "no final response was produced"; no further detail reaches the terminal.
- `agent.log` for this one-shot invocation stops right after plugin discovery — the conversation-turn/API-call entries that normally follow never appear there.
- The real cause was only visible in the auto-generated per-request dump at `~/.hermes/sessions/request_dump_*.json`: `reason: non_retryable_client_error`, `error.type: AuthError`, `401 Invalid API key`.
- The Authorization header actually sent is not the real credential — it is `mask_secret()`'s *display-only* masked rendering of the unresolved `openshell:resolve:env:OPENCODE_ZEN_API_KEY` placeholder string, confirmed by exact reproduction (see Debug Output).
- Workaround confirmed working: invoke with a single `-m "/"` argument instead of separate `--provider`/`-m` flags — same credential, same target, succeeds and returns a correct response.

### Description

Invoking a canonical custom provider (e.g. `opencode-zen`) via separate `--provider` and `-m` CLI flags fails to authenticate. The credential resolution for this code path never substitutes the real secret for the `openshell:resolve:env:` placeholder stored in `.env`; instead, the raw placeholder string appears to get passed through the display-only `mask_secret()` helper (normally used by `hermes config`/`status`/`dump` for safe display), and that masked output is sent as the literal Authorization bearer token. The origin naturally rejects it with 401, which surfaces to the user only as an opaque "no final response was produced".

Expected: either the real credential resolves correctly for this flag combination, or resolution failure produces a clear, actionable error instead of silently sending a masked placeholder as if it were a real key.

### Reproduction Steps

13:52 2026/07/22
1. Register a canonical provider (e.g. `opencode-zen`) with a real API key stored via the `openshell:resolve:env:` placeholder in `.env` (`OPENCODE_ZEN_API_KEY=openshell:resolve:env:OPENCODE_ZEN_API_KEY`).
2. Run: `nemoclaw exec -- hermes -z "test" --provider opencode-zen -m nemotron-3-ultra-free`
3. Observe: `hermes -z: no final response was produced; treating the run as failed.`
4. Inspect `~/.hermes/sessions/request_dump_.json` for the matching timestamp — `reason` is `non_retryable_client_error`, `error.type` is `AuthError`, `error.message` contains `401 ... Invalid API key`.
5. In that same dump, `request.headers.Authorization` is 22 characters. Locally computing `mask_secret("openshell:resolve:env:OPENCODE_ZEN_API_KEY", head=8, tail=4)` produces `"openshel..._KEY"` — concatenated as `"Bearer openshel..._KEY"`, this is an exact match for the value actually sent.
6. Compare: same session, same credential, same model, invoked instead as `nemoclaw exec -- hermes -z "test" -m "opencode-zen/nemotron-3-ultra-free"` (no `--provider` flag) — succeeds, returns a correct response.

### Environment

- OS: Windows 11 + WSL2 (Ubuntu)
- NemoClaw: v0.0.79 (source), OpenShell 0.0.72
- Docker: sandbox container `nemoclaw-sandbox-local:my-hermes-*`, 2 days uptime
- hermes 0.17.0, openai_sdk 2.24.0 (from `hermes dump`)
- Credential registered via standard `openshell provider create/update --credential` using the `openshell:resolve:env:` placeholder pattern

### Debug Output

`nemoclaw debug --quick --sandbox my-hermes` (relevant excerpts, hostname redacted):

```text
═══ System ═══
Linux 6.6.114.1-microsoft-standard-WSL2 #1 SMP PREEMPT_DYNAMIC x86_64 GNU/Linux

═══ Docker ═══
CONTAINER ID IMAGE STATUS NAMES
a01e1f15ba67 nemoclaw-sandbox-local:my-hermes- Up 4 hours openshell-my-hermes-

═══ OpenShell ═══
Server Status
Gateway: nemoclaw-8990
Status: Connected
Version: 0.0.72
NAME CREATED PHASE
my-hermes 2026-07-19 06:48:37 Ready
Sandbox:
Phase: Ready
Policy source: sandbox
Revision: 12
```

Local reproduction of the mask leak (no real secret involved — input is the literal, non-sensitive placeholder text, not a resolved credential):

```python
>>> from agent.redact import mask_secret
>>> mask_secret("openshell:resolve:env:OPENCODE_ZEN_API_KEY", head=8, tail=4)
'openshel..._KEY'
```

`hermes_cli/config.py`'s `load_env()` does a plain `line.partition('=')` parse of `.env` with no handling of the `openshell:resolve:env:` prefix — real substitution is expected to happen before `get_env_value()` reads it. For this provider's `--provider`/`-m`-flags path, that substitution does not happen, so `get_env_value()` falls through to the raw file parse and returns the literal unresolved placeholder — which then appears to reach `mask_secret()` and gets used as the literal credential instead of triggering a clean resolution error.

### Logs

```shell
`~/.hermes/sessions/request_dump_.json` (values illustrative except the Authorization derivation above, which is exact):

{
"timestamp": "2026-07-22T04:01:05.868029",
"reason": "non_retryable_client_error",
"request": {
"method": "POST",
"url": "https://opencode.ai/zen/v1/chat/completions",
"headers": {
"Authorization": "Bearer openshel..._KEY",
"Content-Type": "application/json"
}
},
"error": {
"type": "AuthError",
"message": "Error code: 401 - {'type': 'error', 'error': {'type': 'AuthError', 'message': 'Invalid API key.'}}",
"status_code": 401
}
}

`agent.log` for this invocation shows plugin discovery completing normally, then nothing further — no conversation-turn or API-call entries at all, unlike the long-running gateway process's log. The `request_dump` file was the only place this failure's real cause was visible.
```

### Checklist

- [x] I confirmed this bug is reproducible
- [x] I searched existing issues and this is not a duplicate

Contributor guide

Open the contributing guide

Research direction

Start in hermes_cli/config.py, tracing load_env() and get_env_value() for the separate --provider/-m path, then inspect agent.redact.mask_secret and reproduce the request using the commands and request_dump described here. Done means the real credential is used for this flag combination, or unresolved credentials produce a clear actionable error rather than a masked placeholder reaching Authorization.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, authentication, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.