NVIDIA / NVIDIA/NemoClaw

Phase 2 slice 8: Migrate inference routing to a generated gRPC client

Open
#9,828 0 comments 0 reactions 1 assignee Claimed by @rsliter View on GitHub
area: architecture area: e2e area: inference area: routing integration: openclaw
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

> [!WARNING]
> this issue is pending scoping and should only be picked up by rsliter.

Parent Epic: #9802

## Outcome

Use a release-pinned generated gRPC client for inference route operations that satisfy the Phase 1 inference contract.

## Draft scope

- Generate the TypeScript inference client from the exact `inference.proto` selected by Phase 2 Slice 1.
- Make generation reproducible and keep generated and protobuf wire types inside the OpenShell implementation boundary.
- Map released route get, set, delete, and bundle RPCs used by current consumers to NemoClaw-owned types.
- Preserve recorded-route, live-route, provider, model, endpoint, verification, and route-drift decisions.
- Reuse the reviewed gateway, TLS, authentication, timeout, cancellation, and redaction controls.
- Run shared inference-route contracts against CLI and gRPC implementations.

## Acceptance evidence

- Equivalent route reads and mutations return the same provider, model, endpoint, and changed or unchanged decisions.
- Missing provider, invalid model, rejected route, wrong scope, timeout, and transport failure remain distinct.
- Provider credentials remain in OpenShell gateway custody and do not enter generated messages, errors, logs, or artifacts.
- A timed-out route mutation re-reads the live route before retry, rollback, or cleanup.
- Inference route reachability and validation-request results retain their existing meanings.

## Test plan

- Deterministic protobuf generation and package-contract tests.
- Shared CLI and gRPC inference-route contracts.
- gRPC tests for conversion, status mapping, authentication, timeout, cancellation, redaction, and route drift.
- Focused live evidence for route mutation, inference route reachability, validation requests, and cleanup.

## Dependencies and sequencing

Blocked by Phase 2 Slice 1. This slice can use CLI-managed providers while the provider-management slice remains in progress.

## Deferred scope

- New inference providers, models, or API families.
- Gateway provider-management migration.
- Claims of compatibility or support beyond the existing inference contract.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.