docker / docker/cli

compose parser fails 3.4+ schema for `volume` with `external`, `driver`, `driver_opts`, `labels`

Open
#2,272 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

impact/compose-file
Dominant language
Go
Stars
6.1k
Forks
2.2k
Avg merge
1d 15h
Merged PRs (30d)
43

Description

Description

The compose parser in /cli/compose/loader/ does not follow the compose 3.4+ schema rules as they apply to volumes that are external and also declare driver, driver_opts, labels. Instead, the code only follows the rules that apply for schemas 3.0-3.3.

Errant code has been isolated and trivial fix is ready.

Setup
  • https://github.com/docker/cli 18.06, 18.09, or current master in 19.03
  • go code that loads/parses a compose file using high-level functions like loader.Load() or the low-level errant function loader.LoadVolumes().
  • parse a docker-compose.yml that is version=3.4, has a top-level volumes: that declares a volume with both external:true and driver, driver_opts, or labels
Repro
  1. Setup as above
  2. Parse a docker-compose similar to the following
    version: '3.4'
    volumes:
      myVol:
        external: true
        driver: 'rexray/ebs:0.11.4'
    services:
      web:
        image: ubuntu
        volumes:
          - myVol:/mnt/one
    
Result

Error. conflicting parameters "external" and driver specified for volume myVol

Expected

No error. And to have a fully parsed Volume section.

Notes

The errant code is below. You can see the old 3.0-3.3 rules of mutual exclusion are being applied to all schema versions. The fix is to only apply them to versions 3.0-3.3 as per Docker documentation https://docs.docker.com/compose/compose-file/#external

https://github.com/docker/cli/blob/52714e413cd1f58d5f3734191ad23a4701a9d8e3/cli/compose/loader/loader.go#L579-L595

This is related to old work that didn't implement the full set of rules for 3.4+. Issues https://github.com/docker/cli/issues/274 and https://github.com/docker/cli/issues/608

Trivial fix. I have a commit that I'm using which was based on the 18.09.5 release. Also trivial to apply to the 18.06 and 19.03 releases. Here is the fix https://github.com/diablodale/cli/commit/1d4c6d8e492c0598c432314be76e5f59d4c60386

Happy to make three separate PRs with fixes and test cases for the three branches 18.06, 18.09, and 19.03.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in cli/compose/loader/loader.go around lines 579-595 and reproduce the version 3.4 volume case using loader.Load() or loader.LoadVolumes(). Add coverage for external volumes with driver, driver_opts, and labels; done means the compose file parses without a conflict error and produces a fully parsed Volume section.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
cli
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.