Makefile generator creates misplaced build artifacts for dependencies outside of `binding.gyp` root dir
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 155
- Forks
- 101
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 4
Description
A binding.gyp with dependencies that live outside of its directory will create misplaced artifacts with the makefile generator (usually, on mac & linux). The dependency will be resolved to a relative path with parent directory segments (../../*) and those get joined as-is to the build directory (./build/../../*), resulting in build files outside of the build directory. This result may confuse other tooling too, unfortunately.
For example, the following code in the makefile generator produces a path like ./build/../../node_modules/node-addon-api/node_addon_api.target.mk:
Motivating example
I have a multi-workspace project set up like this:
super/(root)package.json(defines"workspaces")workspaces/native/package.jsonbinding.gyp
super has no dependencies, and native only depends on node-gyp and node-addon-api. Note that in this setup, npm hoists node-addon-gyp to the root node_modules/.
node-addon-api recommends adding it as a dependency in binding.gyp:
# super/workspaces/native/binding.gyp
{
"targets": [{
"target_name": "native",
"sources": ["module.cc"],
"dependencies": ["<!(node -p \"require('node-addon-api').targets\"):node_addon_api"]
}]
}
After running node-gyp rebuild, we end up with some mysterious build artifacts:
super/(root)node_modules/... (real, created by npm)workspaces/node_modules/(not real, created by gyp)node-addon-api/node_addon_api_except_all.target.mknode_addon_api_except.target.mknode_addon_api_maybe.target.mknode_addon_api.target.mknode_addon_api.Makefile
A real issue that this causes is npm list failing:
$ npm ls
super@0.0.1 /[...]/super
├─┬ native@0.0.1 -> ./workspaces/native
│ ├── bindings@1.5.0
│ ├── node-addon-api@ invalid: "^8.5.0" from workspaces/native
│ └── node-gyp@12.1.0
└── node-addon-api@8.5.0 extraneous
npm error code ELSPROBLEMS
npm error extraneous: node-addon-api@8.5.0 /[...]/node_modules/node-addon-api
npm error invalid: node-addon-api@ /[...]/workspaces/node_modules/node-addon-api
# exit code 1
We also see some misplaced files within the build directory too:
super/workspaces/native/build/node_modules/(should be withinRelease/obj.target/)node-addon-api/node_addon_api.stamp
Release/...
I haven't found these artifacts to cause issues, but I am highlighting them because they share the same root cause.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pylib/gyp/generator/make.py at the dependency path handling around lines 2433-2436, then reproduce the issue with the described multi-workspace binding.gyp and node-gyp rebuild setup. Done means dependencies outside the binding.gyp directory no longer create makefiles, stamps, or other artifacts outside the intended build locations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100