Support ref+<backend>:// secret resolution in environment values (vals/helmfile URI syntax)
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 38.2k
- Forks
- 5.8k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 55
Description
Description
When running services that need secrets (database passwords, API keys, etc.), there's no native way to resolve them from external secret stores like OpenBao, HashiCorp Vault, AWS Secrets Manager, or others. Users currently rely on external preprocessors:
vals eval -f compose.yaml | docker compose -f - up
or shell variable substitution, which adds friction and breaks Docker Compose UX (docker compose ps, logs, etc. with stdin-based flows).
Describe the solution you'd like
Support the ref+<backend>://path#/key URI scheme in environment values, consistent with the convention already established by:
- helmfile/vals — the reference implementation of this URI scheme
- ArgoCD Vault Plugin
- helmfile
Example compose.yaml:
services:
postgres:
image: postgres:16
environment:
POSTGRES_USER: "ref+openbao://secret/data/prod/postgres#/username"
POSTGRES_PASSWORD: "ref+openbao://secret/data/prod/postgres#/password"
POSTGRES_DB: "ref+openbao://secret/data/prod/postgres#/database"
myapp:
image: myapp:latest
environment:
DATABASE_URL: "ref+vault://secret/data/prod/postgres#/connection_string"
API_KEY: "ref+awssecrets://prod/api-keys#/stripe"
depends_on:
- postgres
The URI format follows the vals convention:
ref+<backend>://<path>#/<key>
| Component | Description |
|---|---|
ref+ |
Prefix indicating a secret reference (distinguishes from regular values) |
<backend> |
Secret store backend (openbao, vault, awssecrets, gcpsecrets, azurekeyvault, etc.) |
<path> |
Path to the secret in the backend |
#/<key> |
JSON pointer to extract a specific field from the secret |
Correlation with helmfile/vals
The vals project defines this URI scheme and supports 20+ backends. Docker Compose could either:
- Use vals as a library (
github.com/helmfile/vals) — immediate support for all backends with minimal code - Implement the same URI scheme natively — starting with a few common backends (OpenBao, Vault, AWS Secrets Manager) using a pluggable resolver interface, keeping compatibility with the vals syntax so users have a consistent experience across tools
Either way, adopting the same ref+<backend>:// syntax means users who already use helmfile, ArgoCD Vault Plugin, or vals get a familiar, zero-learning-curve experience in Docker Compose.
Proposed implementation
Resolution would happen at project load time (after interpolation, before container creation), in pkg/compose/loader.go:
- Values prefixed with
ref+are detected and resolved against the appropriate backend - Backend authentication uses standard environment variables (
OPENBAO_ADDR/OPENBAO_TOKEN,VAULT_ADDR/VAULT_TOKEN,AWS_REGION, etc.) - Non-
ref+values are untouched (fully backward compatible) - A resolver interface makes adding new backends straightforward
Describe alternatives you've considered
- vals eval as preprocessor: Works but breaks normal Docker Compose workflows (can't use named projects,
docker compose down, etc. properly with stdin) - Shell variable substitution with wrapper scripts: Exposes secrets in process environment and shell history
- Docker secrets / external secret operators: Only works with Swarm or Kubernetes, not standalone Compose
Additional context
- The
ref+prefix makes this opt-in and backward compatible — existing compose files are unaffected - OpenBao and HashiCorp Vault use nearly identical APIs (OpenBao is a fork), so supporting both is trivial
- This would make Docker Compose a first-class citizen in secret-store-backed workflows alongside Helm and ArgoCD
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with pkg/compose/loader.go and trace project-load interpolation through container creation. Compare the requested ref+ URI handling and vals-compatible backend choices, including resolver authentication, with the existing loader entry points. Done means opt-in references resolve at load time while ordinary values remain unchanged, with behavior covering the proposed backends.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker-compose, go
- Domain
- backend, devops, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100