concourse / concourse/docker-image-resource

get_params: save does not save image in v5.0.0

Open
#263 7 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

needs-investigation
Dominant language
Go
Stars
162
Forks
250
PR merge metrics
No merged PRs in 30d

Description

I have a multi-stage docker build that I am trying to get caching to work with but when I put the docker resource with get_params: {save: true} it appears that the image is deleted from the folder and is not saved on the next build step.

I am running concourse v5.0.0.

Here is my setup:


resources:

  - name: repo-release
    type: git
    source:
      uri: {{github-repo-uri}}
      branch: release
      private_key: {{github-private-key}}

  - name: frontend-image-base
    type: docker-image
    source: ((redacted))
  - name: frontend-image-builder
    type: docker-image
    source: ((redacted))
  - name: frontend-image
    type: docker-image
    source: ((redacted))

# JOBS
jobs:
  - name: build-frontend
    plan:
      - get: repo
        resource: repo-release
        trigger: true

      - get: frontend-image-builder
        inputs: # use blank inputs to decrease container file sizes
        params:
          save: true
      - get: frontend-image
        inputs:
        params:
          save: true
      - get: frontend-image-base
        inputs:
        params:
          save: true
        #/ End Aggregate


      # START BUILD (placed in do block for success/fail/cancel handles)
      - do:
        # BUILD BUILDER
        - put: frontend-image-builder
           get_params:
             skip_download: true
             save: true
           params:
             target_name: builder
             build: repo
             cache_from:
               - frontend-image-builder
             load_bases:
               - frontend-image-base

        # BUILDER RUNNER
        - put: frontend-image
           get_params:
             skip_download: true
           params:
             build: repo
             target_name: runner
             load_bases:
               - frontend-image-base
               - frontend-builder-new
             cache_from:
               - frontend-builder-new
               - frontend-image
             tag_as_latest: true
        on_success:
        on_failure:
        on_cancel:

When I run fly inspect on put: frontend-image-builder I can see the image in the frontend-image-builder folder.

When I run fly inspect on put: frontend-image I can only see digest, image-id, and tag in the frontend-image-builder folder. It looks like frontend-image-builder is deleting the image and it is not saving over for some reason. Is this a bug or am I doing something wrong here?

My workaround for the moment is to add a get in between the builder and the frontend which seems to be able to pull the latest frontend-image-builder. For example:

...
# START BUILD
      - do:
        # BUILD BUILDER
        - put: frontend-image-builder
          # same as before
          
          # REGET BUILDER BECAUSE CONCOURSE HAS A BUG?
        - get: frontend-builder-new
          resource: frontend-image-builder
          inputs:
          params:
            save: true

        # BUILDER RUNNER
        - put: frontend-image
           # same as before
            load_bases:
              - frontend-image-base
              - frontend-builder-new # note different tag
            cache_from:
              - frontend-builder-new # note different tag
              - frontend-image
...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the multi-stage Docker build configuration from the issue and compare the docker-image resource's get_params save behavior across the two put steps. Trace why the saved image is not available to the next step without an intervening get. Done means the builder image remains usable by the following build without the workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
40/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.