Internal dependencies treated as external when using npm workspaces
- Dominant language
- Go
- Stars
- 40.1k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
I'm working with a monorepo using npm workspaces. The bundle is targeted for nodejs and the bundling goals are as follows:
* Treat internal dependencies as internal and bundle eligible
* Treat exteral dependencies as external and do not include them in the bundle
With npm workspaces, each package in the workspace has a symlink of the workspace in the root `node_modules` folder pointing to the workspace's directory (that containing the workspace's `package.json`). Therefore a consumer of an internal package can import from the package without using relative pathing and normal node module resolution will proceed up the directory tree until it discovers the module in the root `node_modules` folder. So if you have a workspace with a name of `@foo/bar` a consumer can `import { baz } from '@foo/bar';` and resolution will be successful via the symlink.
Using the [bundling for node](https://esbuild.github.io/getting-started/#bundling-for-node) recommendations, I've marked the root `node_modules` folder as external. However, doing so prevents the bundling of the "internal" dependencies that are symlinked. Although the symlinked folder doesn't have a source path that is in `node_modules`, it is still getting flagged as external using this mechanism.
I attempted the plugin route to try and specifically flag these internal dependencies as not external with something similar to:
```
const checkExternalPlugin = {
name: 'checkExternal',
setup(build) {
build.onResolve({ filter: /@foo\// }, () => ({ external: false }));
},
};
require('esbuild')
.build({
entryPoints: ['src/index.ts'],
platform: 'node',
target: 'node16.16',
bundle: true,
outdir: './dist',
plugins: [checkExternalPlugin],
external: ['./node_modules/*', '../../node_modules/*'],
})
.catch((err) => {
console.error('error in build', err);
process.exit(1);
});
```
The `onResolve` callback fires as expected, but it does not appear that setting `external: false` will prevent the checking of `node_modules` for external.
An interesting note here is that if I remove the external checks for `node_modules` that the `path` which gets to the `onLoad` callback is the source file of the symbolic link unless `preserveSymlinks` is set to `true` in which case then the path is the symlink. That behavior makes sense, but it seems like the external checking should exhibit a similar behavior.
I can't come up with a way to solve for this with a userland plugin, but perhaps there is an approach I am missing. Otherwise, I can think of two ways to resolve this:
1. Allow setting `external: false` in `onResolve` to carry through and override any configuration setting of `external`. My testing seems to indicate that you can explicitly designate something as `external: true` and that will override the lack of configuration for that file being external, but you cannot set something as `external: false` to override configuration including it as external.
2. When checking for externals based on the configuration, use the resolved path that would get passed through to the `onLoad` plugin as the path. That is, if `preserveSymlinks` is false then the check should be against the source path. If `preserveSymlinks` is true then it would be the symlinked path. In my scenario, `preserveSymlinks` would remain false and the internal dependency would not be treated as external.
The final alternative is some plugin approach that I might not be thinking of so if there is some guidance there I'd appreciate it as well.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the npm workspaces case with the shown build configuration, including the nodejs platform, external node_modules patterns, onResolve, onLoad, and preserveSymlinks behavior. Trace how the resolved symlinked package is classified as external; done means internal workspace dependencies are bundled while external dependencies remain excluded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, nodejs, typescript
- Domain
- build-system, tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100