spring-projects / spring-projects/spring-boot

Paketo integration doesn't work with Docker's new overlayfs storage driver

Open
#49,251 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: bug
Dominant language
Java
Stars
81.5k
Forks
42.7k
Avg merge
2d 4h
Merged PRs (30d)
65

Description

Docker Engine 29 switched the default storage backend from overlay2 to overlayfs, see this documentation.

GitHub Actions switched to Docker 29 for ubuntu-latest some time ago. This broke something in Paketo (see #49214), but this has been fixed now.

Our Paketo integration is now in a half-broken state: stock-standard bootBuildImage works, but this doesn't:

bootBuildImage {
    runImage = "paketobuildpacks/ubuntu-noble-run:latest"
}

This results in this error:


    [creator]     Added 5/5 app layer(s)
    [creator]     Adding layer 'buildpacksio/lifecycle:launcher'
    [creator]     Adding layer 'buildpacksio/lifecycle:config'
    [creator]     Adding layer 'buildpacksio/lifecycle:process-types'
    [creator]     Adding label 'io.buildpacks.lifecycle.metadata'
    [creator]     Adding label 'io.buildpacks.build.metadata'
    [creator]     Adding label 'io.buildpacks.project.metadata'
    [creator]     Adding label 'org.opencontainers.image.title'
    [creator]     Adding label 'org.opencontainers.image.version'
    [creator]     Adding label 'org.springframework.boot.version'
    [creator]     Setting default process type 'web'
    [creator]     Saving docker.io/library/demo:0.0.1-SNAPSHOT...
    [creator]     ERROR: failed to export: saving image: failed to fetch base layers: open /tmp/imgutil.local.image.82301454/blobs/sha256/23a8c5303fec5323ef417f9fd39ccaec49bb046a46081aa4670bc05a25e2ae83: no such file or directory

which is the same problem as in our system test. All branches are affected.

Upgrading to the latest Docker version (v29.2.1 at the time of writing) doesn't help.

Switching to overlay2 (the old storage driver) or disabling the snapshots outlined in this comment solves the problem.

My Fedora and my ubuntu-24.04 VM use overlay2 (the old storage driver) by default, maybe because they have upgraded Docker from 28 to 29.

To switch to the new storage driver, create a file named /etc/docker/daemon.json and put this in:

{
  "features": {
    "containerd-snapshotter": true
  }
}

docker info then shows:

 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1

instead of

 Storage Driver: overlay2

But even with those options in place, I can't get the tests to fail on my local machine.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Paketo integration and the linked system test, then reproduce the reported bootBuildImage configuration using Docker 29 with the overlayfs storage driver. Done means the custom runImage build succeeds and the affected system test passes without requiring overlay2 or disabled snapshots.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java, spring-boot
Domain
build-system, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.