NVIDIA / NVIDIA/NemoClaw

On DGX Spark: 2 NemoHermes Agents to 1 vLLM Local Model

Open
#10,036 3 comments 0 reactions 0 assignees View on GitHub
area: local-models integration: hermes needs: info platform: dgx-spark provider: vllm
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

### Investigation Summary

On a dgx spark trying to connect 2 seperate hermes (mine and my sons) to a single shared docker vlmm with gemma.
It's been pain staking where the gapteways or api's start to break and system fights itself when I'm using an architecture that is supposed to be by design. I tried to do it more secure with rootless dockers but it wouldn't work.
Post in case fixes can occur. My claude dgx/hermes troubleshooter project first searches new features, forums and bug fixes before helping me in troubleshooting to help context and if I'm running into a new bug.

### Description

I have a DGX Spark/Nemohermes troubleshooting project/skill in claude that helped with the troubleshooting and write up. I've attached that in the debug form.

### Reproduction Steps

Steps and issues defined in the markdown in the debug output

### Environment

DGX Spark, 2 nemohermes agents that connect to a shared docker with vllm and Gemma. Design in the markdown file in debug output

### Debug Output

[nemoclaw-issue-port-propagation.md](https://github.com/user-attachments/files/31354922/nemoclaw-issue-port-propagation.md)

### Logs

```shell

```

### Checklist

- [x] I confirmed this bug is reproducible
- [x] I searched existing issues and this is not a duplicate

Contributor guide

Open the contributing guide

Research direction

Start with the attached nemoclaw-issue-port-propagation.md and reproduce the setup on a DGX Spark with two NemoHermes agents sharing a Docker vLLM service with Gemma. Trace the gateway and API failures described there; done means both agents can use the shared local model without the reported port-propagation or service conflicts.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
ai, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.