mapbox / mapbox/node-pre-gyp

Error when building in Docker due to layered filesystem

Open
#644 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
1.2k
Forks
271
Avg merge
23h 40m
Merged PRs (30d)
2

Description

Hey! 👋🏻 I'm working on getting Linux ARM builds for https://github.com/TryGhost/node-sqlite3 but I'm running into an issue when using node-pre-gyp in Docker.

#10 [6/8] RUN npx node-pre-gyp build
#10 0.458 node-pre-gyp info it worked if it ends with ok
#10 0.460 node-pre-gyp info using node-pre-gyp@1.0.9
#10 0.460 node-pre-gyp info using node@16.14.2 | linux | arm64
#10 0.485 node-pre-gyp ERR! UNCAUGHT EXCEPTION
#10 0.485 node-pre-gyp ERR! stack Error: EXDEV: cross-device link not permitted, rename 'build-tmp-napi-v3' -> 'build'
#10 0.485 node-pre-gyp ERR! stack     at Object.renameSync (node:fs:980:3)
#10 0.485 node-pre-gyp ERR! stack     at Object.module.exports.swap_build_dir_in (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/util/napi.js:179:8)
#10 0.485 node-pre-gyp ERR! stack     at /usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/build.js:21:12
#10 0.485 node-pre-gyp ERR! stack     at handle_gyp_opts (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/util/handle_gyp_opts.js:101:10)
#10 0.486 node-pre-gyp ERR! stack     at do_build (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/build.js:13:3)
#10 0.486 node-pre-gyp ERR! stack     at build (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/build.js:49:12)
#10 0.486 node-pre-gyp ERR! stack     at Object.self.commands.<computed> [as build] (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/node-pre-gyp.js:86:37)
#10 0.486 node-pre-gyp ERR! stack     at run (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/main.js:81:30)
#10 0.486 node-pre-gyp ERR! stack     at Object.<anonymous> (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/main.js:125:1)
#10 0.486 node-pre-gyp ERR! stack     at Module._compile (node:internal/modules/cjs/loader:1103:14)
#10 0.487 node-pre-gyp ERR! System Linux 5.10.104-linuxkit
#10 0.487 node-pre-gyp ERR! command "/usr/local/bin/node" "/usr/src/build/node_modules/.bin/node-pre-gyp" "build"
#10 0.487 node-pre-gyp ERR! cwd /usr/src/build
#10 0.488 node-pre-gyp ERR! node -v v16.14.2
#10 0.488 node-pre-gyp ERR! node-pre-gyp -v v1.0.9
#10 ERROR: executor failed running [/bin/sh -c npx node-pre-gyp build]: exit code: 7

According to https://docs.docker.com/storage/storagedriver/overlayfs-driver/#modifying-files-or-directories:

Renaming directories: Calling rename(2) for a directory is allowed only when both the source and the destination path are on the top layer. Otherwise, it returns EXDEV error (“cross-device link not permitted”). Your application needs to be designed to handle EXDEV and fall back to a “copy and unlink” strategy.

You should be able to reproduce the issue by cloning the node-sqlite3 repo, dropping the following into tools/BinaryBuilder.Dockerfile:

ARG NODE_VERSION=16
ARG VARIANT=bullseye

FROM node:$NODE_VERSION-$VARIANT

WORKDIR /usr/src/build

COPY . .
RUN npm install --ignore-scripts
RUN npx node-pre-gyp configure
RUN npx node-pre-gyp build
RUN npm run test
RUN npx node-pre-gyp package

CMD ["sh"]

and then running

docker buildx build --file ./tools/BinaryBuilder.Dockerfile --progress plain --no-cache .

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in lib/util/napi.js at swap_build_dir_in, where the reported rename fails, and reproduce it with tools/BinaryBuilder.Dockerfile using the supplied docker buildx command. Check the existing build flow and npm run test; done means the Docker build completes past node-pre-gyp build without the EXDEV error.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, nodejs
Domain
build-system, devtools
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.