Fail fast on an unmapped port, and report the mapping back
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
A vLLM server on an unmapped Vast port starts cleanly, passes its local health
check, and is unreachable from outside. Nothing in the logs says so — the
failure only appears when something tries to route to it.
### Background
Vast maps container ports to public ones **at instance creation**. A port not
declared on the template cannot be added afterwards; the instance has to be
recreated. The external port is assigned from the instance's range and is not
the same number as the internal one:
```
$ env | grep VAST_TCP_PORT
VAST_TCP_PORT_10100=12711
$ echo "$PUBLIC_IPADDR"
80.188.223.202
```
So the server is reachable at `80.188.223.202:12711`, and `10100` appears
nowhere outside the box.
### Three changes
**1. Refuse to launch on an unmapped port.**
`install_set.py` should read `VAST_TCP_PORT_` before writing the
Supervisor unit. If it is absent, fail with a message naming the port rather
than starting a server nobody can reach.
This is the highest-value part. The current failure is silent and looks like
success.
**2. Force `--host 0.0.0.0` in the generated unit.**
The Vast template's own vLLM launches with `--host 127.0.0.1`:
```
vllm serve Qwen/Qwen3.5-9B ... --host 127.0.0.1 --port 18000
```
Bound to loopback, it is unreachable from outside regardless of how the port is
mapped — and it looks identical to a mapping problem from the outside. Worth
setting explicitly rather than inheriting whatever the template does.
**3. Report the public address back to llmao.**
The box knows `PUBLIC_IPADDR` and its `VAST_TCP_PORT_*` map at exactly the
moment it fetches `GET /vllm/config`. Having `install_set.py` POST them back
means llmao never has to be told:
```json
POST /vllm/observed
{ "public_ip": "80.188.223.202",
"port_map": { "10100": 12711 },
"name": "gemma4-26b" }
```
llmao can then build `api_base` for the route itself. This is an extension of
the observed-state reporting already planned, not a new mechanism.
### Also worth doing
**Declare ports on the template, not per instance.** Every box then gets them
without anyone remembering, and a missing port becomes a template fix rather
than a rebuild.
### Note for anything pinning the address
The external port differs per instance, so a CDN backend or hiera value pinned
to `IP:port` needs updating on every rebuild. Worth asking whether the CDN can
point at a hostname we control instead, so a rebuild becomes a DNS update
rather than a ticket.
### Reproducing
```bash
# on the box -- is vLLM listening?
ss -tlnp | grep 10100
curl -sS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:10100/v1/models
# from outside -- is the mapping routing?
curl -sS -m 10 -o /dev/null -w '%{http_code}\n' http://$PUBLIC_IPADDR:/v1/models
```
Local works and external times out → the port was not declared at creation.
Both fail → vLLM is not up, or it is bound to loopback.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in install_set.py and trace how the Supervisor unit is generated and where GET /vllm/config is handled. Use the supplied ss and curl commands to reproduce local-versus-external reachability, then inspect the existing observed-state reporting before adding the requested observation fields. Done means unmapped ports fail before launch, the generated service is externally reachable, and llmao receives the public address and port map.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, cloud, devops, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100