webpack / webpack/webpack

Webpack doesn’t work well with wasm modules created with Emscripten

Open
#7,352 16 comments 46 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: wasm
Dominant language
JavaScript
Stars
65.9k
Forks
9.5k
Avg merge
10h 40m
Merged PRs (30d)
296

Description

Feature request

What is the current behavior?
The modularized JS emitted by Emscripten registers a global with a given name that loads the wasm file on invocation, initializes the wasm runtime and returns a Module.

Making it work with Webpack is quite hard as there seems to be interference with Webpack 4 defaults.
This is the webpack.config.js that I came up with:

const webpack = require("webpack");
const path = require("path");

module.exports = {
  mode: "development",
  entry: "./index.js",
  output: {
    path: path.resolve(__dirname, "dist"),
    filename: "bundle.js"
  },
  module: {
    defaultRules: [
      {
        type: "javascript/auto",
        resolve: {}
      }
    ],
    rules: [
      {
        test: /fibonacci\.js$/,
        loader: "exports-loader"
      },
      {
        test: /fibonacci\.wasm$/,
        loader: "file-loader",
        options: {
          publicPath: "dist/"
        }
      }
    ]
  },
  // This is necessary due to the fact that emscripten puts both Node and
  // web code into one file. The node part uses Node’s `fs` module to load
  // the wasm file.
  // Issue: https://github.com/kripken/emscripten/issues/6542.
  plugins: [new webpack.IgnorePlugin(/(fs)/)]
};

(Here is a minimal test project in a gist that you can clone and build with npm start. Docker required!)

edit:
In the meantime, @sokra informed that that I can simplify the configuration a bit (and make it less like a sledgehammer):

module.exports = {
  /* ... */
  browser: { 
    "fs": false // ← !!
  },
  module: {
    rules: [
      /* ... */
      {
        test: /fibonacci\.wasm$/,
        type: "javascript/auto", // ← !!
        loader: "file-loader",
        options: {
          publicPath: "dist/"
        }
      }
    ]
  },
};

Unexpected things I had to do

  • I needed to overwrite defaultRules as otherwise some sort of default rule will run in addition to the ones I specified and making webpack error “Module parse failed: magic header not detected” (try it!)
  • I needed to specify file-loader for the wasm file as otherwise webpack tries to resolve the names of the wasm module’s import object like env, which are provided by the JS file.
  • I needed to set a locateFile() function as webpack changes the file (and potentially path) of the wasm file and Emscripten hardcodes that name (not visible here but in the gist)

I am not sure what the right course of action here is, but considering that most wasm projects are going to be built with Emscripten, I feel like it’s worth making it easier.

Happy to answer Qs to give y’all a clearer picture.

What is the expected behavior?

Ideally, Webpack would recognize the typical Emscripten JS files and automatically bundle the accomodating wasm module and make paths work.

Other relevant information:
webpack version: 4.8.3
Node.js version: 10
Operating System: Mac OS 10.13.4
Additional tools:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked minimal test project and its webpack.config.js, index.js, fibonacci.js, and fibonacci.wasm; run npm start to reproduce the current setup. Compare the required defaultRules, file-loader, and locateFile configuration with Webpack’s wasm handling. Done means a typical Emscripten output is bundled automatically and its wasm path works without this custom configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, wasm, webpack
Domain
build-system, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.