slimtoolkit / slimtoolkit/slim

Getting error `API error (404): Could not find the file /opt/dockerslim/artifacts/creport.json in container <CONTAINER_ID>` on all my builds all of a sudden

Open
#320 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
23.4k
Forks
840
PR merge metrics
No merged PRs in 30d

Description

All of a sudden I've started getting docker-slim-related errors during my builds, no matter if I revert to the exact same steps I've followed until the errors started happening.

Here is an excerpt from my build logs:

time="2022-05-06T13:43:45Z" level=error msg="ipc.Client.GetEvent(): event channel error = EOF\n"
time="2022-05-06T13:43:45Z" level=warning msg="docker-slim: warning" error=EOF stack="goroutine 1 [running]:\nruntime/debug.Stack(0x0, 0xc0003b1650, 0x53a305)\n\truntime/debug/stack.go:24 +0x9f\ngithub.com/docker-slim/docker-slim/pkg/util/errutil.WarnOn(0x10c8460, 0xc000084030)\n\tgithub.com/docker-slim/docker-slim/pkg/util/errutil/errutil.go:38 +0x54\ngithub.com/docker-slim/docker-slim/pkg/app/master/inspectors/container.(*Inspector).FinishMonitoring(0xc0003fb600)\n\tgithub.com/docker-slim/docker-slim/pkg/app/master/inspectors/container/container_inspector.go:1166 +0x3b1\ngithub.com/docker-slim/docker-slim/pkg/app/master/commands/build.OnCommand(0xc0000ae7e0, 0xc000033680, 0x7ffdf1c20eab, 0x53, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)\n\tgithub.com/docker-slim/docker-slim/pkg/app/master/commands/build/handler.go:1280 +0x42fd\ngithub.com/docker-slim/docker-slim/pkg/app/master/commands/build.glob..func1(0xc0000a0b40, 0x7a, 0x100)\n\tgithub.com/docker-slim/docker-slim/pkg/app/master/commands/build/cli.go:636 +0x339d\ngithub.com/urfave/cli/v2.(*Command).Run(0x16dade0, 0xc0005c4980, 0x0, 0x0)\n\tgithub.com/urfave/cli/v2@v2.3.0/command.go:163 +0x4ed\ngithub.com/urfave/cli/v2.(*App).RunContext(0xc0000a4000, 0x10dfdc0, 0xc000098010, 0xc0000a2000, 0x7, 0x7, 0x0, 0x0)\n\tgithub.com/urfave/cli/v2@v2.3.0/app.go:313 +0x81f\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/v2@v2.3.0/app.go:224\ngithub.com/docker-slim/docker-slim/pkg/app/master.Run()\n\tgithub.com/docker-slim/docker-slim/pkg/app/master/app.go:15 +0x6c\nmain.main()\n\tgithub.com/docker-slim/docker-slim/cmd/docker-slim/main.go:15 +0x54\n" version="linux|Transformer|1.37.4|2e1992528f556df7e5f91109aef813f0d4488404|2022-02-28_05:50:22AM"
time="2022-05-06T13:43:45Z" level=error msg="channel.CommandClient.Call: c.Read error = EOF"
time="2022-05-06T13:43:45Z" level=error msg="ipc.Client.SendCommand() cmd channel call error=EOF\n"
time="2022-05-06T13:43:45Z" level=error msg="dockerutil.CopyFromContainer: dclient.DownloadFromContainer() error = API error (404): Could not find the file /opt/dockerslim/artifacts/creport.json in container d8b7350a1cece0101dbba165b4a17a306fc4e08251a0db14995b4ea28eca0b98"
time="2022-05-06T13:43:45Z" level=fatal msg="docker-slim: failure" error="API error (404): Could not find the file /opt/dockerslim/artifacts/creport.json in container d8b7350a1cece0101dbba165b4a17a306fc4e08251a0db14995b4ea28eca0b98" stack="goroutine 1 [running]:\nruntime/debug.Stack(0xc00003c000, 0xc0003b1718, 0x419c85)\n\truntime/debug/stack.go:24 +0x9f\ngithub.com/docker-slim/docker-slim/pkg/util/errutil.FailOn(0x10c8880, 0xc0003e1860)\n\tgithub.com/docker-slim/docker-slim/pkg/util/errutil/errutil.go:28 +0x54\ngithub.com/docker-slim/docker-slim/pkg/app/master/inspectors/container.(*Inspector).ShutdownContainer(0xc0003fb600, 0x4, 0xc0003b2308)\n\tgithub.com/docker-slim/docker-slim/pkg/app/master/inspectors/container/container_inspector.go:1077 +0x67a\ngithub.com/docker-slim/docker-slim/pkg/app/master/commands/build.OnCommand(0xc0000ae7e0, 0xc000033680, 0x7ffdf1c20eab, 0x53, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)\n\tgithub.com/docker-slim/docker-slim/pkg/app/master/commands/build/handler.go:1283 +0x436f\ngithub.com/docker-slim/docker-slim/pkg/app/master/commands/build.glob..func1(0xc0000a0b40, 0x7a, 0x100)\n\tgithub.com/docker-slim/docker-slim/pkg/app/master/commands/build/cli.go:636 +0x339d\ngithub.com/urfave/cli/v2.(*Command).Run(0x16dade0, 0xc0005c4980, 0x0, 0x0)\n\tgithub.com/urfave/cli/v2@v2.3.0/command.go:163 +0x4ed\ngithub.com/urfave/cli/v2.(*App).RunContext(0xc0000a4000, 0x10dfdc0, 0xc000098010, 0xc0000a2000, 0x7, 0x7, 0x0, 0x0)\n\tgithub.com/urfave/cli/v2@v2.3.0/app.go:313 +0x81f\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/v2@v2.3.0/app.go:224\ngithub.com/docker-slim/docker-slim/pkg/app/master.Run()\n\tgithub.com/docker-slim/docker-slim/pkg/app/master/app.go:15 +0x6c\nmain.main()\n\tgithub.com/docker-slim/docker-slim/cmd/docker-slim/main.go:15 +0x54\n" version="linux|Transformer|1.37.4|2e1992528f556df7e5f91109aef813f0d4488404|2022-02-28_05:50:22AM"

I'm using docker-slim do minify my final images inside of Google Cloud Build. Here is an example cloudbuild.yaml I'm using:

substitutions:
    _PROJECT_IMAGE_NAME: "gcr.io/<REDACTED>"

options:
    dynamic_substitutions: true

steps:
    # Build the container image
    - name: "gcr.io/cloud-builders/docker"
      args:
          [
              "build",
              "-t",
             "${_PROJECT_IMAGE_NAME}:${COMMIT_SHA}",
              "-f",
              "pipelines/Dockerfile.gcp.staging.api",
              ".",
          ]

    # Minify the container image
    - name: "gcr.io/cloud-builders/docker"
      entrypoint: "bash"
      args:
          - "-c"
          - |
              curl -L -o ds.tar.gz https://downloads.dockerslim.com/releases/1.37.6/dist_linux.tar.gz;
              tar -xvf ds.tar.gz;
              mv  dist_linux/docker-slim /usr/local/bin/;
              mv  dist_linux/docker-slim-sensor /usr/local/bin/;
              docker-slim build --sensor-ipc-mode proxy --sensor-ipc-endpoint $(docker network inspect bridge -f '{{range .IPAM.Config}}{{.Gateway}}{{end}}' | cut -f1) ${_PROJECT_IMAGE_NAME}:${COMMIT_SHA};
    
    # Push the minified container image to Container Registry
    - name: "gcr.io/cloud-builders/docker"
      args: ["push", "${_PROJECT_IMAGE_NAME}:${COMMIT_SHA}.slim"]
    
    # Deploy container image to Cloud Run
    - name: "gcr.io/cloud-builders/gcloud"
      args:
          - "alpha"
          - "run"
          - "deploy"
          - "<REDACTED>" # <- Cloud Run service name
          - "--image"
          - "${_PROJECT_IMAGE_NAME}:${COMMIT_SHA}.slim"
          - "--region"
          - "europe-west8"
          - "--project"
          - "vas-01"
          - "--execution-environment"
          - "gen2"

images:
    - "${_PROJECT_IMAGE_NAME}:${COMMIT_SHA}.slim"

UPDATE:
I'm still trying to run my builds. After ~10 repeated fails, now it's sometimes working. 99% of my builds are still broken though.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with the cloudbuild.yaml steps and the docker-slim 1.37.6 binary. Trace the reported failures through pkg/app/master/inspectors/container/container_inspector.go, especially FinishMonitoring and ShutdownContainer, and the build entry point in pkg/app/master/commands/build/handler.go. Done means repeated Cloud Build runs no longer fail because /opt/dockerslim/artifacts/creport.json is missing.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go, google-cloud
Domain
build-system, cloud, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.