NVIDIA / NVIDIA/NemoClaw

[Hermes] OpenShell 0.0.106 endpoint binding regresses Langfuse credentials

Open
#10,840 0 comments 0 reactions 0 assignees View on GitHub
area: integrations area: sandbox area: security bug integration: hermes
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## User Story

As a NemoClaw Hermes operator, I want Langfuse credentials to remain in OpenShell providers and resolve only for the configured Langfuse endpoint, so observability continues working without placing raw secrets in the sandbox.

## Problem Statement

The provider-backed Langfuse setup that worked before the OpenShell 0.0.106 migration no longer emits traces on current NemoClaw releases. Hermes sees revision-scoped `openshell:resolve:env:v...` values, while raw Langfuse keys written into the sandbox still work.

PR #7447 fixed the earlier Hermes-side validator failure by accepting exact `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` resolver placeholders. That patch is still present in NemoClaw v0.0.115 and current main. The regression occurs later in the request path.

NemoClaw v0.0.112 moved its managed OpenShell baseline from 0.0.101 to 0.0.106 in PR #9192. OpenShell 0.0.106 includes NVIDIA/OpenShell#2510, which changed static credential resolution from destination-independent placeholder rewriting to provider-identity and endpoint-bound resolution:

- a profile with endpoints resolves its static credentials only at those declared endpoints;
- an endpointless profile requires an explicit sandbox-policy `credential_binding` naming the attached provider instance;
- an endpointless, unbound provider has its static credential handles withheld;
- a request outside the bound host, port, or path fails closed.

NemoClaw did not migrate Langfuse across that contract change. It has the Hermes validator patch, but no managed Langfuse provider profile, no base-URL-derived endpoint binding in the sandbox policy, no provider publication ordering for that binding, and no live trace-ingestion qualification. The old manual path can therefore reach Hermes with placeholders but cannot reliably resolve them on egress under OpenShell 0.0.106.

This is a regression rather than only a missing feature: a provider-backed workflow reported working on older releases stopped working after the credential-resolution contract tightened, while the unsafe raw-secret path continued to work.

Related design issue: #7446.

## Impact / Why This Matters

- Hermes Langfuse observability stops working after upgrading or rebuilding onto the OpenShell 0.0.106 runtime path.
- The only reported workaround is to persist raw Langfuse credentials inside the sandbox, bypassing NemoClaw's credential-custody boundary.
- The existing image-build test proves only that Hermes accepts the placeholder syntax. It does not prove provider attachment, endpoint authorization, Basic-auth rewriting, trace ingestion, or secret sanitization.
- Operators can mistake a visible placeholder for a Hermes parsing failure when the actual denial is OpenShell's endpoint-bound credential gate.

## Regression Boundary

- Reported working: releases using the earlier OpenShell credential-resolution behavior.
- Contract change entered NemoClaw: v0.0.112 through PR #9192, upgrading OpenShell 0.0.101 to 0.0.106.
- Reported failing: v0.0.115 and later after upgrade or sandbox rebuild.
- Hermes placeholder compatibility patch: still present in v0.0.115 and current main.

The v0.0.115 observation is consistent with the sandbox first being recreated on the newer runtime or provider state. The patch was not removed in v0.0.115.

## Reproduction Steps

1. Use NemoClaw v0.0.115 or later with Hermes and OpenShell 0.0.106.
2. Configure valid `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` values in an OpenShell v2 provider.
3. Attach the provider to the Hermes sandbox and configure the Langfuse plugin with a cloud or self-hosted base URL.
4. Confirm Hermes sees revision-scoped `openshell:resolve:env:v...` values rather than raw keys.
5. Enable `observability/langfuse` and restart or rebuild Hermes.
6. Generate an agent turn.
7. Observe that no trace arrives and inspect the OpenShell denial or unresolved-credential failure.
8. Replace the placeholders with raw keys inside the sandbox and repeat. Trace emission succeeds, demonstrating that the remaining failure is the managed credential path.

## Expected Result

NemoClaw creates or validates a Langfuse credential provider, binds that exact provider instance to the configured Langfuse host, port, and required ingestion paths before Hermes starts, and emits a trace without exposing raw keys inside the sandbox.

## Actual Result

Hermes can accept the placeholder syntax, but the provider-backed request is not guaranteed to have an authorized endpoint binding. OpenShell withholds or rejects the credential, so the Langfuse request fails. Raw sandbox credentials work because they bypass OpenShell credential binding.

## Proposed Remediation

- Add a checked-in, exact Langfuse credential profile for `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY`.
- Derive a bounded HTTPS endpoint from the configured Langfuse base URL and bind the exact attached provider instance in the sandbox policy before sandbox creation or rebuild.
- Ensure Hermes consumes the process-injected standard Langfuse variables and does not persist a stale revision-scoped placeholder that can shadow the current provider environment.
- Preserve raw-key prefix validation and exact same-name placeholder validation.
- Fail closed for non-HTTPS remote URLs, malformed or credentialed URLs, mismatched provider identity, missing bindings, unresolved placeholders, and redirects outside the configured origin.
- Add a live E2E that proves authenticated trace ingestion through OpenShell, endpoint containment, and absence of raw keys from sandbox-visible files, environment output, logs, and debug artifacts.
- Document the supported cloud and self-hosted configuration flow and remove the claim that the validator patch alone completes compatibility.

## Acceptance Criteria

- [ ] No raw Langfuse secret is stored in a sandbox-visible file or process environment.
- [ ] `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` arrive as exact revision-scoped OpenShell placeholders in the Hermes process.
- [ ] The configured Langfuse HTTPS origin is present in the effective policy with `credential_binding.provider` naming the exact attached Langfuse provider.
- [ ] A real Hermes turn produces a trace at the configured Langfuse instance through OpenShell 0.0.106.
- [ ] The same placeholders fail closed at every non-bound host, port, and path.
- [ ] Provider rotation and sandbox rebuild retain a usable current revision without a stale `~/.hermes/.env` override.
- [ ] Tests distinguish Hermes local validation failure from OpenShell endpoint-binding or credential-projection failure.
- [ ] User documentation includes the supported setup and redacted diagnostics.

## Environment

- NemoClaw: reported from v0.0.115; current main remains affected
- Hermes: managed 0.19.0 path on v0.0.115 through v0.0.118
- OpenShell: 0.0.106
- Langfuse: cloud or self-hosted HTTPS endpoint

## Related Work

- #7446 tracks the broader supported-integration contract.
- #7447 added the narrow Hermes placeholder validator compatibility patch.
- #9192 migrated NemoClaw to OpenShell 0.0.106.
- NVIDIA/OpenShell#2510 introduced endpoint-bound static credential resolution.

Contributor guide

Open the contributing guide

Research direction

Start with PR #9192, PR #7447, and the existing image-build test to trace the current Hermes placeholder flow, then inspect the OpenShell provider and sandbox-policy setup described in the issue. Done means a managed Langfuse provider is endpoint-bound, a live Hermes turn ingests a trace, non-bound requests fail closed, and tests and documentation cover secret sanitization and supported setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
documentation, observability, security, testing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.