Compose: allowlisted backend environment variables for inference process

Summary

docker model compose up can pass context-size, runtime-flags and speculative-decoding settings into ConfigureBackend, but it cannot pass process environment variables for the backend (for example GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 for AMD UMA workarounds).

The runner already supports this at the process layer: RunnerConfig.Env is validated by ValidateEnv and appended onto os.Environ() when spawning the backend. Compose and BackendConfiguration never expose a field for it, so compose users cannot set backend env without leaving the compose provider path.

Why this matters

• Use cases like llama.cpp UMA / unified memory need env vars on the inference process (see also #994).
• Ollama-style compose snippets set env on the service container. DMR compose only configures the model backend and cannot mirror that today.
• Maintainers have already indicated env should not be exposed as an unauthenticated REST knob for security reasons. Compose is the right control plane because it is local provider configuration, not a broad network API.

Current code shape (main)

Proposed direction (for discussion)

1. Add an optional Environment []string (KEY=VALUE entries) to BackendConfiguration (or under LlamaCpp / shared block) and plumb it into RunnerConfig.Env when starting runners.
2. On docker model compose up, add a string-array flag such as --env KEY=VALUE (and/or a documented allowlist of keys).
3. Prefer an allowlist for keys (e.g. GGML_*, selected CUDA/Vulkan vars) so compose cannot inject arbitrary process env into the backend. Reject unknown keys with a clear error.
4. Advertise the parameter via compose metadata so Compose integration can document it.
5. Keep REST/OpenAI inference paths from accepting free-form env from remote clients.

Alternatives already available

• Manual llama-server after docker model pull (works, loses compose lifecycle).
• Global process env for the whole model-runner daemon (too coarse and easy to misconfigure).

Related

• #994 (UMA / unified memory env example; this issue is the compose/API plumbing to set such vars safely)

I am happy to implement an allowlisted prototype if this direction sounds right.

Start in cmd/cli/commands/compose.go and trace how compose up and compose metadata build BackendConfiguration. Then follow BackendConfiguration into pkg/inference/backends, where RunnerConfig.Env and ValidateEnv already exist. Done means compose can pass only approved environment entries, metadata advertises the option, unknown keys are rejected clearly, and REST paths remain unable to set free-form backend environment.