docker / docker/compose

Support ref+<backend>:// secret resolution in environment values (vals/helmfile URI syntax)

Open
#13,821 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/feature status/0-triage
Dominant language
Go
Stars
38.2k
Forks
5.8k
Avg merge
2d 14h
Merged PRs (30d)
55

Description

Description

When running services that need secrets (database passwords, API keys, etc.), there's no native way to resolve them from external secret stores like OpenBao, HashiCorp Vault, AWS Secrets Manager, or others. Users currently rely on external preprocessors:

vals eval -f compose.yaml | docker compose -f - up

or shell variable substitution, which adds friction and breaks Docker Compose UX (docker compose ps, logs, etc. with stdin-based flows).

Describe the solution you'd like

Support the ref+<backend>://path#/key URI scheme in environment values, consistent with the convention already established by:

Example compose.yaml:

services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_USER: "ref+openbao://secret/data/prod/postgres#/username"
      POSTGRES_PASSWORD: "ref+openbao://secret/data/prod/postgres#/password"
      POSTGRES_DB: "ref+openbao://secret/data/prod/postgres#/database"

  myapp:
    image: myapp:latest
    environment:
      DATABASE_URL: "ref+vault://secret/data/prod/postgres#/connection_string"
      API_KEY: "ref+awssecrets://prod/api-keys#/stripe"
    depends_on:
      - postgres

The URI format follows the vals convention:

ref+<backend>://<path>#/<key>
Component Description
ref+ Prefix indicating a secret reference (distinguishes from regular values)
<backend> Secret store backend (openbao, vault, awssecrets, gcpsecrets, azurekeyvault, etc.)
<path> Path to the secret in the backend
#/<key> JSON pointer to extract a specific field from the secret

Correlation with helmfile/vals

The vals project defines this URI scheme and supports 20+ backends. Docker Compose could either:

  1. Use vals as a library (github.com/helmfile/vals) — immediate support for all backends with minimal code
  2. Implement the same URI scheme natively — starting with a few common backends (OpenBao, Vault, AWS Secrets Manager) using a pluggable resolver interface, keeping compatibility with the vals syntax so users have a consistent experience across tools

Either way, adopting the same ref+<backend>:// syntax means users who already use helmfile, ArgoCD Vault Plugin, or vals get a familiar, zero-learning-curve experience in Docker Compose.

Proposed implementation

Resolution would happen at project load time (after interpolation, before container creation), in pkg/compose/loader.go:

  • Values prefixed with ref+ are detected and resolved against the appropriate backend
  • Backend authentication uses standard environment variables (OPENBAO_ADDR/OPENBAO_TOKEN, VAULT_ADDR/VAULT_TOKEN, AWS_REGION, etc.)
  • Non-ref+ values are untouched (fully backward compatible)
  • A resolver interface makes adding new backends straightforward

Describe alternatives you've considered

  • vals eval as preprocessor: Works but breaks normal Docker Compose workflows (can't use named projects, docker compose down, etc. properly with stdin)
  • Shell variable substitution with wrapper scripts: Exposes secrets in process environment and shell history
  • Docker secrets / external secret operators: Only works with Swarm or Kubernetes, not standalone Compose

Additional context

  • The ref+ prefix makes this opt-in and backward compatible — existing compose files are unaffected
  • OpenBao and HashiCorp Vault use nearly identical APIs (OpenBao is a fork), so supporting both is trivial
  • This would make Docker Compose a first-class citizen in secret-store-backed workflows alongside Helm and ArgoCD

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with pkg/compose/loader.go and trace project-load interpolation through container creation. Compare the requested ref+ URI handling and vals-compatible backend choices, including resolver authentication, with the existing loader entry points. Done means opt-in references resolve at load time while ordinary values remain unchanged, with behavior covering the proposed backends.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker-compose, go
Domain
backend, devops, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.