docker / docker/compose

docker-compose dynamic volume names for replicas

Open
#9,026 10 comments 7 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/feature
Dominant language
Go
Stars
38.2k
Forks
5.8k
Avg merge
2d 14h
Merged PRs (30d)
55

Description

moved over from https://github.com/moby/moby/issues/43079

Tried some of the "solutions" from this thread and stackoverflow. No swarm, node or other hosts at all. Just trying to get it to work locally first without starting containers around docker-compose. Main idea is to get rid of these copies and just use replicas:

  configsvr0:
    image: mongo
    command: mongod --configsvr --replSet configsvr --port 27017 --dbpath /data/db --keyFile /data/keyfile
    volumes:
      - configsvr0:/data/db
      - ./keyfile:/data/keyfile:ro
  configsvr1:
    image: mongo
    command: mongod --configsvr --replSet configsvr --port 27017 --dbpath /data/db --keyFile /data/keyfile
    volumes:
      - configsvr1:/data/db
      - ./keyfile:/data/keyfile:ro
  configsvr2:
    image: mongo
    command: mongod --configsvr --replSet configsvr --port 27017 --dbpath /data/db --keyFile /data/keyfile
    volumes:
      - configsvr2:/data/db
      - ./keyfile:/data/keyfile:ro

volumes:
  configsvr0:
  configsvr1:
  configsvr2:

See the difference? It's just counting up on the volume name...

It would be great to just have something like this:

  configsvr0:
    image: mongo
    command: mongod --configsvr --replSet configsvr --port 27017 --dbpath /data/db --keyFile /data/keyfile
    volumes:
      - configsvr:/data/db
      - ./keyfile:/data/keyfile:ro

volumes:
  configsvr:
    name: 'configsvr-{{.Task.Slot}}'

Tried this (see below) but it didn't work.

Note: Below I only copied small junks from my yml that I thought are important for this.

test 1 - volume name

would require to add entries under volumes so that's not really dynamic - but I gave it a try:

    volumes:
      - "configsvr{{.Task.Slot}}:/data/db"
➜  msc git:(develop) ✗ docker-compose up configsvr0
service "configsvr0" refers to undefined volume configsvr{{.Task.Slot}}: invalid compose project
test2 - dynamic name with task slot

that would be dynamic but still doesn't work though it was accepted in stackoverflow. That's actually my sample from top.

    volumes:
      - configsvr:/data/db

volumes:
  configsvr:
    name: 'configsvr-{{.Task.Slot}}'
➜  msc git:(develop) ✗ docker-compose up configsvr0
[+] Running 0/0
 ⠿ Volume "configsvr-{{.Task.Slot}}"  Error                                                                                                                                                          0.0s
Error response from daemon: create configsvr-{{.Task.Slot}}: "configsvr-{{.Task.Slot}}" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path
my env

mac monteray with docker desktop.

➜  msc git:(develop) ✗ docker -v
Docker version 20.10.11, build dea9396
➜  msc git:(develop) ✗ docker-compose -v
Docker Compose version v2.2.1

docker-compose.yml version: 3.9

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the two documented examples with Docker Compose v2.2.1, using the provided docker-compose.yml snippets and the reported docker-compose up configsvr0 commands. Read the Compose handling for service replicas and volume names; done means a single declared volume can produce valid per-replica names without requiring copied service or volume entries.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, docker-compose, yaml
Domain
devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.