NVIDIA / NVIDIA/NemoClaw

[Managed inference] Decide whether managed llama.cpp may use a configured host port

Open
#9,644 0 comments 0 reactions 0 assignees View on GitHub
area: local-models area: providers needs: design
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## Summary

Decide whether the NemoClaw-managed llama.cpp runtime may use a configured host port instead of always reserving host port 8081.

The accepted managed llama.cpp contract in #8144 requires both first-class llama.cpp paths to use reserved port 8081 and fail on a conflict. A new `NEMOCLAW_LLAMACPP_PORT` setting would change that contract and needs a separate product decision.

## Current behavior

- The managed llama.cpp server uses host port 8081.
- Port conflicts stop onboarding.
- The serving recipe, runtime receipt, policy, route, recovery, status, and cleanup paths assume the reserved port contract.

## Proposed product decision

Allow `NEMOCLAW_LLAMACPP_PORT` to select the host listener while preserving the container-side and request-guard contract.

Default behavior remains port 8081. NemoClaw must not select another port automatically.

## Lifecycle and compatibility boundary

- Validate the configured port through the shared port authority before managed side effects.
- Reject conflicts with the dashboard range, gateway, vLLM, Ollama, runtime adapters, and other configured services.
- Persist the selected port in the managed runtime receipt and recovery state.
- Reuse the exact selected port across resume, status, restart, recover, rebuild, and route reconciliation.
- Reject receipt or runtime drift instead of silently returning to 8081.
- Cleanup must target only the NemoClaw-owned runtime identified by the receipt.
- Existing-server attachment remains a separate contract unless maintainers include it in this decision.

## Security boundary

- Preserve loopback-only host exposure and native bearer authentication.
- Do not expose the listener on LAN or public interfaces.
- Keep the request guard and upstream container port ownership explicit.
- Stop before process, container, credential, policy, provider, or sandbox mutation when validation fails.

## Validation boundary

- Port parsing and collision tests.
- Managed install, receipt, status, restart, recovery, and cleanup tests.
- Policy and route tests for the selected port.
- Negative tests that prove no process or persistent state is created after rejection.
- Physical managed llama.cpp E2E on each activated platform.

## Acceptance criteria

- [ ] Maintainers decide whether configured host ports fit the #8144 product contract.
- [ ] Maintainers define whether existing-server attachment also accepts the setting.
- [ ] Port 8081 remains the default.
- [ ] A configured port persists across every accepted lifecycle boundary.
- [ ] Invalid or conflicting ports stop before effects.
- [ ] Loopback exposure and bearer authentication remain enforced.
- [ ] Documentation states the default, override, persistence, and recovery behavior.

Related: #684, #8144, and PR #9537.

Signed-off-by: Prekshi Vyas

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the accepted managed llama.cpp contract in #8144 and the related PR #9537, then map the serving recipe, runtime receipt, recovery, status, route, and cleanup paths named here. The first step is a maintainer decision on configured ports and existing-server attachment; done means an accepted contract, lifecycle and validation tests, preserved security boundaries, and documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, infrastructure, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.