live runner: capacity is per registration, and only one GPU can be described

Two related gaps in the live runner registry, both surfacing when one container serves several capabilities. Filing together because they share a root cause: the registry models advertisements, not machines.

1. Capacity is tracked per registration, so shared hardware is overcommitted

Capacity lives on the runner and is checked against that runner's own session map:

if len(runner.sessions) >= runner.Capacity {
    return "", "", &RunnerError{StatusCode: http.StatusServiceUnavailable, Message: "no capacity available for runner"}
}

ai/runner/live_runner.go:973-974, against runners map[string]*liveRunner (:280).

Two register_runner calls produce two runner_ids and therefore two independent capacity counters. normalizeHeartbeat (:857-900) validates each request in isolation and never looks for other runners sharing a runner_url, so registering ollama/qwen and ollama/llama against the same http://ollama:11434, each with capacity: 1, advertises two slots backed by one process and one GPU.

The GPU field does not help: it is stored (:850), cloned (:889), and republished in discovery (:1722), but appears nowhere in ReserveSession or any admission path. Two runners reporting the same gpu.id remain unrelated capacity pools.

Discovery propagates the inflated number, CapacityAvailable: runner.Capacity - len(runner.sessions) (:1729), and server/remote_discovery.go:275-277 zeroes CapacityAvailable on both sides before comparing, so it is not even a distinguishing field there.

Failure mode: with one GPU and two capabilities at capacity: 1, the orchestrator admits a session on the second while the first saturates the GPU. Both callers paid, both were admitted, and contention shows up as latency or OOM inside the container rather than the 503 that would have been correct.

Suggested fix: an explicit capacity pool

One optional field on the heartbeat request and the static config entry, opaque and operator-chosen:

Pool         string `json:"pool,omitempty"`          // runners sharing this share capacity
PoolCapacity int    `json:"pool_capacity,omitempty"` // limit for the pool as a whole

Two independent limits: capacity keeps its per-runner meaning (at most one llama generation at a time) while pool_capacity bounds the shared resource (at most two generations on this card). Members of a pool must agree on pool_capacity; a mismatch is a registration error, which catches config drift rather than silently picking a winner.

Admission checks the specific limit first, then the shared one:

if len(runner.sessions) >= runner.Capacity { → 503 "no capacity available for runner" }
if pool != nil && pool.used >= pool.capacity { → 503 "no capacity available for pool" }

Notes for whoever implements it:

• Release is where the bugs will be. Every path that drops a session must decrement the pool: ReleaseSession, the payment-failure release at server/ai_http.go:341-357, and especially the expiryLoop at :1369-1380 reaping a dead runner that still holds sessions. Missing that leaks pool slots permanently, which is worse than the bug being fixed.
• Keep pool counters under the existing registry lock rather than adding a third mutex; the current order is r.mu then runner.mu, and a separate pool lock invites an inversion.
• Discovery must report min(runner.Capacity - len(runner.sessions), pool.capacity - pool.used), or the overcommit survives at the advertisement layer. Exposing pool in the discovery entry also lets a client see that two runners are siblings.
• Backward compatible: absent pool means the runner is its own singleton pool, i.e. today's behaviour exactly.

Why not derive the pool from gpu.id: it is unsafe as a key (see part 2), and a two-GPU container cannot be described by the field at all, so any GPU-derived capacity is wrong by construction.

Workaround until then, for app authors: hold one semaphore per container sized to the real hardware, flip every registration's status to non-ready together when it is exhausted, and push a heartbeat on the transition instead of waiting for the 5s timer. Note capacity: 0 is not expressible, since normalizeHeartbeat coerces it to 1 (:885-887); the lever is status, which isReadyStatus (:1819-1821) gates in ReserveSession.

2. Only one GPU can be described, and gpu.id is not consistently unique

LiveRunnerGPU is a single struct, and every field carrying it is singular:

type LiveRunnerGPU struct {
	ID     string `json:"id,omitempty"`
	Name   string `json:"name,omitempty"`
	VRAMMB int    `json:"vram_mb,omitempty"`
}

ai/runner/live_runner.go:76-80, used as GPU *LiveRunnerGPU at :136, :155, :191.

The SDK mirrors it: detect_process_gpu() returns the first single result from three detectors, and _detect_gpu_pynvml resolves one device index (the process's own, else the first visible) and returns one object. So a container with two cards reports exactly one, and vram_mb describes half the machine.

Separately, the two producers of gpu.id disagree on uniqueness. The SDK uses nvmlDeviceGetUUID, which is globally unique. go-livepeer's liveRunnerGPUForIndex uses ID: fmt.Sprintf("%d", deviceIndex) (:695-696), a bare "0" that collides across hosts and across containers on the same card.

Suggested fix: add GPUs []LiveRunnerGPU alongside the existing singular field for compatibility, and make the go-side id carry something stable (UUID or PCI address) rather than a device index.

Independent of part 1 — the pool does not need this, and this does not fix the pool.

Start in ai/runner/live_runner.go with normalizeHeartbeat, ReserveSession, ReleaseSession, expiryLoop, and discovery generation, then trace the payment-failure release in server/ai_http.go and comparisons in server/remote_discovery.go. Inspect the SDK's detect_process_gpu and _detect_gpu_pynvml paths as well as liveRunnerGPUForIndex. Done means shared capacity is enforced and released across registrations, discovery reflects it, and multiple GPUs have stable identifiers without breaking the singular field.