Define the experimental voice gateway contract v0.0.1
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Parent
- #8275
## Related work
- #8378 delivered the first authenticated, committed-turn gateway slice.
- #8482 defined the current OpenClaw response projection behavior.
- #9235 moved startup credentials to fixed inherited descriptors.
- #9411 and PR #9412 own OpenClaw context reuse across sequential admissions.
- #8403 owns deployment-identity provisioning, rotation, recovery, and removal.
## Problem
The experimental voice gateway has a tested HTTP and NDJSON boundary, but the boundary has no explicit contract version. Runtime integrations pin NemoClaw revisions instead of verifying a named compatibility contract.
The implemented behavior is also distributed across source, tests, and earlier issues. Maintainers do not have one reviewable definition of the first gateway MVP.
This issue defines the first experimental contract as `0.0.1`. It does not establish a supported surface.
## Outcome
A trusted host-side voice runtime can verify and use the NemoClaw voice gateway contract `0.0.1` to:
1. Check the authenticated loopback gateway and its contract version.
2. Create one short-lived voice session.
3. Submit one finalized text turn.
4. Receive correlated NDJSON response events and one terminal outcome.
5. Close the session and revoke its grant.
6. Reuse OpenClaw context through separate sequential admissions for one runtime conversation.
The gateway owns authentication, input validation, trusted agent binding, OpenClaw credential custody, limits, failure classification, diagnostics, and cleanup.
## Contract boundary
```text
trusted host-side voice runtime
-> authenticated loopback HTTP
NemoClaw voice gateway
-> authenticated loopback WebSocket
OpenClaw agent gateway
```
The runtime-facing contract begins with finalized text. It ends with projected assistant text.
The contract does not include audio, WebRTC, RTVI, speech recognition, speech synthesis, voice activity detection, playback, or UI events.
## Contract version
NemoClaw defines:
```text
VOICE_GATEWAY_CONTRACT_VERSION=0.0.1
NemoClaw-Voice-Gateway-Contract-Version: 0.0.1
```
The authenticated `GET /healthz` response and every authenticated API response must include the version header.
Every state-changing request must include the same header. The gateway must authenticate the request before it validates the version. The gateway must validate the version before it parses the body or changes session or agent state.
An absent or different version returns:
```http
HTTP/1.1 412 Precondition Failed
Content-Type: application/json
NemoClaw-Voice-Gateway-Contract-Version: 0.0.1
{"error":"contract_version_mismatch","supportedVersion":"0.0.1"}
```
Version `0.0.1` is immutable after acceptance. A wire or semantic change requires a new contract version. The `/v1/voice/...` prefix identifies the API family. It does not establish 1.0 stability or product support.
## Deployment contract
The gateway remains an internal, feature-gated foreground process:
```text
nemoclaw internal voice-gateway serve
--gateway-url
--runtime-identity
--runtime-profile
--sandbox
--agent
[--listen-port ]
```
The process requires:
```text
NEMOCLAW_EXPERIMENTAL_VOICE_GATEWAY=1
```
The gateway listens on `127.0.0.1:18800` by default. The OpenClaw URL must use `ws://`, a literal loopback address, an explicit port, and the exact `/ws` path.
The trusted launcher supplies two distinct credentials:
- File descriptor `3`: runtime deployment bearer.
- File descriptor `4`: OpenClaw bearer.
The credential sources must remain current-user-owned regular files that are inaccessible to group and other users. The launcher must not place credential values or paths in the child arguments or environment.
## Runtime-facing HTTP API
### Health check
```http
GET /healthz
Authorization: Bearer
```
Success returns `204` and the contract-version header.
### Create a session
```http
POST /v1/voice/sessions
Authorization: Bearer
NemoClaw-Voice-Gateway-Contract-Version: 0.0.1
Content-Type: application/json
{"runtimeConversationId":"conversation-123"}
```
Success returns:
```json
{
"voiceSessionId": "gateway-generated-id",
"grant": "session-scoped-bearer",
"expiresAt": "RFC-3339 timestamp"
}
```
### Commit one turn
```http
POST /v1/voice/sessions/{voiceSessionId}/turns
Authorization: Bearer
NemoClaw-Voice-Gateway-Contract-Version: 0.0.1
Content-Type: application/json
{"commitId":"runtime-generated-id","text":"finalized user transcript"}
```
Success returns `application/x-ndjson`.
### Close a session
```http
DELETE /v1/voice/sessions/{voiceSessionId}
Authorization: Bearer
NemoClaw-Voice-Gateway-Contract-Version: 0.0.1
```
Success returns `204`.
Request JSON objects are closed. The gateway rejects additional or missing fields.
## Response event contract
A successful response contains:
```text
response.started
zero or more response.text.delta events
response.completed
```
Each event contains the same `voiceSessionId`, `turnId`, and `responseId`. Text sequence numbers start at `0` and increase by one.
A failed response ends with `response.failed`. A failure before OpenClaw accepts the turn may contain only the terminal event.
Terminal failure reasons are:
```text
agent_failed
agent_gateway_unavailable
agent_protocol_error
response_too_large
session_closed
session_expired
turn_timeout
```
Exactly one terminal event ends each admitted response. No event follows a terminal event.
The wire permits incremental text events. Contract `0.0.1` does not require token-latency streaming and may return one complete text event.
## Session and context contract
Contract `0.0.1` permits:
- One active voice session per gateway process.
- One committed turn per admitted session.
- One OpenClaw request per accepted commit.
- Sequential admissions only.
- A five-minute session lifetime.
- A two-minute turn timeout.
- A 48 KiB finalized-text limit.
- A 2 MiB response-text limit.
The runtime must close a terminal session before it creates another session.
For separate admissions, the same trusted binding and `runtimeConversationId` must select the same private OpenClaw context. A changed binding value must select a different context.
The binding contains:
```text
agent
runtimeProfile
runtimeIdentity
sandbox
runtimeConversationId
```
The runtime cannot provide an OpenClaw session key, agent, sandbox, gateway URL, or OpenClaw credential. NemoClaw derives the private OpenClaw session key and never returns it through HTTP, NDJSON, or diagnostics.
Closing a voice session closes one admission. It does not delete OpenClaw history or guarantee cancellation of work already accepted by OpenClaw.
## Request errors
Before NDJSON streaming starts, errors use `{"error":"error_code"}`.
Contract `0.0.1` includes:
```text
authentication_failed
contract_version_mismatch
duplicate_turn
invalid_request
session_expired
session_in_progress
session_not_found
turn_in_progress
turn_limit_reached
internal_error
```
The existing HTTP status mappings remain unchanged. `contract_version_mismatch` uses `412`.
## Diagnostics
The gateway emits content-free JSON diagnostics for gateway, session, and turn lifecycle changes.
Diagnostics must not contain:
- Transcript or response text.
- Deployment or OpenClaw credentials.
- Session grants.
- Private OpenClaw session keys.
- Native OpenClaw frames.
The runtime must treat `runtimeConversationId` as a non-sensitive correlation identifier because diagnostics record it.
## Acceptance criteria
- [ ] The authenticated health response advertises exact contract version `0.0.1`.
- [ ] Each authenticated API response includes the contract-version header.
- [ ] A state-changing request with version `0.0.1` preserves the current response and error behavior.
- [ ] A missing or different version fails with `412` before body parsing, session mutation, or OpenClaw connection.
- [ ] Invalid authentication fails before version and body validation.
- [ ] One authenticated runtime completes one committed turn through the HTTP and pinned OpenClaw boundaries.
- [ ] Separate admissions for one runtime conversation retain OpenClaw context after #9411 lands.
- [ ] A different conversation or trusted binding cannot access that context.
- [ ] Duplicate or overlapping work does not invoke OpenClaw twice.
- [ ] Expiry, explicit close, timeout, oversized output, and protocol errors produce the specified terminal result.
- [ ] Tests confirm that credentials, grants, private session keys, transcripts, responses, and native frames do not cross forbidden boundaries.
- [ ] Process-level package tests confirm the version header and credential-descriptor contract through the compiled CLI.
- [ ] The issue records the exact VoiceClaw and OpenClaw revisions used for composed compatibility evidence.
## Non-goals
- Product support for the voice gateway or a voice runtime.
- A public voice protocol.
- Audio, media, or provider behavior in NemoClaw.
- Parallel sessions or turns.
- Response supersession or agent-generation cancellation.
- Reconnect, resume, or durable conversation storage.
- Human-user authentication.
- Deployment-identity lifecycle work from #8403.
- Voice-runtime installation, start, supervision, or restart.
- ElevenLabs Speech Engine integration.
- A new dependency or runtime-profile framework.
This issue does not complete the cancellation, managed lifecycle, second-runtime, or support requirements in #8275.
## Category
Feature
## Checklist
- [x] I searched open and closed issues.
- [x] This issue does not duplicate the implementation owned by #9411 or #8403.
- [x] I described the proposed contract, security boundaries, and acceptance evidence.
Contributor guide
Research direction
Start at the `nemoclaw internal voice-gateway serve` entry point and trace the existing authenticated HTTP and NDJSON gateway behavior. Read the process-level package tests and the related work in #9411 and #9412 before mapping the acceptance criteria to implementation and compatibility evidence. Done means the version, security boundaries, lifecycle behavior, errors, and composed revisions are covered by tests and recorded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, authentication, backend, security, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100