[Managed inference] Decide whether managed llama.cpp may use a configured host port
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Summary
Decide whether the NemoClaw-managed llama.cpp runtime may use a configured host port instead of always reserving host port 8081.
The accepted managed llama.cpp contract in #8144 requires both first-class llama.cpp paths to use reserved port 8081 and fail on a conflict. A new `NEMOCLAW_LLAMACPP_PORT` setting would change that contract and needs a separate product decision.
## Current behavior
- The managed llama.cpp server uses host port 8081.
- Port conflicts stop onboarding.
- The serving recipe, runtime receipt, policy, route, recovery, status, and cleanup paths assume the reserved port contract.
## Proposed product decision
Allow `NEMOCLAW_LLAMACPP_PORT` to select the host listener while preserving the container-side and request-guard contract.
Default behavior remains port 8081. NemoClaw must not select another port automatically.
## Lifecycle and compatibility boundary
- Validate the configured port through the shared port authority before managed side effects.
- Reject conflicts with the dashboard range, gateway, vLLM, Ollama, runtime adapters, and other configured services.
- Persist the selected port in the managed runtime receipt and recovery state.
- Reuse the exact selected port across resume, status, restart, recover, rebuild, and route reconciliation.
- Reject receipt or runtime drift instead of silently returning to 8081.
- Cleanup must target only the NemoClaw-owned runtime identified by the receipt.
- Existing-server attachment remains a separate contract unless maintainers include it in this decision.
## Security boundary
- Preserve loopback-only host exposure and native bearer authentication.
- Do not expose the listener on LAN or public interfaces.
- Keep the request guard and upstream container port ownership explicit.
- Stop before process, container, credential, policy, provider, or sandbox mutation when validation fails.
## Validation boundary
- Port parsing and collision tests.
- Managed install, receipt, status, restart, recovery, and cleanup tests.
- Policy and route tests for the selected port.
- Negative tests that prove no process or persistent state is created after rejection.
- Physical managed llama.cpp E2E on each activated platform.
## Acceptance criteria
- [ ] Maintainers decide whether configured host ports fit the #8144 product contract.
- [ ] Maintainers define whether existing-server attachment also accepts the setting.
- [ ] Port 8081 remains the default.
- [ ] A configured port persists across every accepted lifecycle boundary.
- [ ] Invalid or conflicting ports stop before effects.
- [ ] Loopback exposure and bearer authentication remain enforced.
- [ ] Documentation states the default, override, persistence, and recovery behavior.
Related: #684, #8144, and PR #9537.
Signed-off-by: Prekshi Vyas
Contributor guide
Research direction
Start by reviewing the accepted managed llama.cpp contract in #8144 and the related PR #9537, then map the serving recipe, runtime receipt, recovery, status, route, and cleanup paths named here. The first step is a maintainer decision on configured ports and existing-server attachment; done means an accepted contract, lifecycle and validation tests, preserved security boundaries, and documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, infrastructure, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100