nodejs / nodejs/node

Tracking Issue: Syncify the ESM Loader

Open
#55,782 13 comments 11 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

esm loaders never-stale performance
Dominant language
JavaScript
Stars
122k
Forks
37.3k
Avg merge
4d 2h
Merged PRs (30d)
283

Description

The code under lib/internal/modules/esm, a.k.a. the ESM loader, contains many functions that are async. We should refactor as many of these as possible, ideally all of them, to be synchronous. This should improve the performance of evaluating ESM code, bringing it roughly on par with the speed of running CommonJS code.

Longer term, once the ESM loader is synchronous and we land the synchronous module customization hooks, we could deprecate monkey-patching the CommonJS loader and merge together the CommonJS and ESM loaders, eliminating duplication: https://github.com/nodejs/node/issues/50356.

This issue will track our progress syncifying the various files and functions of the ESM loader until we can get as much of it to be as synchronous as possible.

The files to be updated, all underlib/internal/modules:
  • run_main.js: asyncRunEntryPointWithESMLoader
  • esm/fetch_module.js: fetchWithRedirects
  • esm/fetch_module.js: isLocalAddress
  • esm/hooks.js: Hooks class (the async methods here probably don’t need updating as they will be removed once we migrate to the synchronous customization hooks)
  • esm/hooks.js: nextHookFactory
  • esm/load.js: getSource https://github.com/nodejs/node/pull/57419
  • esm/load.js: defaultLoad https://github.com/nodejs/node/pull/57419
  • esm/loader.js: ModuleLoader.eval
  • esm/loader.js: ModuleLoader.getModuleJobForImport
  • esm/loader.js: ModuleLoader.loadAndTranslate
  • esm/loader.js: ModuleLoader.import
  • esm/loader.js: ModuleLoader.load
  • esm/module_job.js: ModuleJob._link
  • esm/module_job.js: ModuleJob._instantiate
  • esm/module_job.js: ModuleJob.run
  • esm/module_job.js: ModuleJobSync.run
  • esm/translators.js: wasm handler, via translators.set('wasm', ...
  • esm/utils.js: importModuleDynamicallyCallback
  • esm/utils.js: initializeHooks (might not need updating as we will remove this once the synchronous customization hooks land
  • esm/worker.js: customizedModuleWorker (might not need updating as we will remove this once the synchronous customization hooks land
  • esm/worker.js: handleMessage (might not need updating as we will remove this once the synchronous customization hooks land

@nodejs/loaders @mcollina @JakobJingleheimer @joyeecheung

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the unchecked entries in lib/internal/modules, especially the named functions under esm/loader.js and esm/module_job.js, and read the linked synchronous module customization hooks proposal. Determine which async functions remain in scope and how their callers depend on them. Done means converting an appropriate set of listed functions to synchronous operation while preserving the ESM loader behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend, devtools
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.