nodejs / nodejs/require-in-the-middle

Removal of resolve module fallback in v8.0.0 can cause resolution issues in (webpack) bundled code

Open
#120 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
186
Forks
34
PR merge metrics
No merged PRs in 30d

Description

Context

Recently a fallback on the resolve module was removed in: https://github.com/nodejs/require-in-the-middle/commit/67076ff3b0dc16c3db71de335285e07ed1137dc1#diff-e727e4bdf3657fd1d798edcd6b099d6e092f8573cba266154583a746bba0f346L73-L82 This was part of the 8.0.0 release.

This fallback seems not to have been intended to support resolving external modules from inside bundled code, but as a side effect it actually did. When bundling code with Webpack, at runtime Webpack provides its own loader instead of the node.js native require.resolve. In short this means that when using require.resolve inside code bundled with Webpack, it can only resolve modules that are contained in the bundle. So external modules can't possibly be resolved this way.

The above linked commit basically removes a fallback to the resolve module for when (require.resolve && require.resolve.paths) doesn't evaluate to truthy. Webpack doesn't have require.resolve.paths on its "stubbed" require.resolve since it isn't useful / needed for the way that module resolution works in the Webpack runtime. So the check would fail since the require.resolve.paths is undefined (and thus falsy). This lead to the resolve module being used instead, with which you were able to actually resolve those external modules.

The reason I found this was because we did some dependency upgrades in the https://github.com/open-telemetry/opentelemetry-lambda layer for nodejs. Upgrading instrumentation libraries that use RITM indirectly through @opentelemetry/instrumentation. The latter upgraded the RITM version from 7.1.1 to 8.0.0. And now since the code for the node.js layer is bundled with Webpack, that means RITM inside our bundle will be actually using Webpack's stubbed require.resolve. This results in breaking instrumentation that needs to hook any non-core modules through RITM. Because those modules need to be external (not included in the bundle) for instrumentation libraries to be able to detect them being loaded.

Solutions

We have temporarily fixed this in the lambda layer by externalizing RITM from the webpack bundle (https://github.com/open-telemetry/opentelemetry-lambda/pull/2037), but would like to see if any alternative solutions are possible.

I suppose the removal of the resolve module was a very deliberate choice to reduce footprint / increase performance?

One solution I could think of would be to have some check for when ritm is running inside a webpack bundle, pseudo-code but could look something like this:

function resolve (moduleName, basedir) {
    let _resolve = function (moduleName, basedir) {
        return require.resolve(moduleName, { paths: [basedir] })
    }
    if (webpackDetected) {
        _resolve = function (moduleName, basedir) {
            return __non_webpack_require__.resolve(moduleName, { paths: [basedir] })
        }
    }
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the resolve fallback changed in commit 67076ff3b0dc16c3db71de335285e07ed1137dc1 and examine how require.resolve behaves in Webpack-bundled Node.js code. Reproduce the failure with an external module that must be detected by instrumentation. Done means bundled consumers can resolve such external modules without requiring RITM to be externalized from the bundle.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, webpack
Domain
backend, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.