rust-lang / rust-lang/rust

std::autodiff fails in forward-over-reverse in wasm

Open
#161,110 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

C-bug F-autodiff O-wasm
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

Using std::autodiff forward-over-reverse fails in wasm (The following is a codex explanation):

  1. Reverse Enzyme differentiates the dynamic loop.
  2. Reverse mode must cache intermediate loop values for the backward traversal.
  3. On wasm32, this Enzyme version emits those cache allocations with a 64-bit size.
  4. WASM’s allocator expects a 32-bit size_t.
  5. LLVM lowers the incompatible call to a .Lmalloc_bitcast_invalid trap.

On x86-64, the allocation size and pointer ABI are both 64-bit, so the original reverse
gradient plus forward-over-reverse HVP works.

MWE (Created with codex):

main.rs:

#![feature(autodiff)]

use std::autodiff::autodiff_reverse;

const OP_VAR: u32 = 0;
const OP_MUL: u32 = 1;
const OP_SIN: u32 = 2;
const OP_ADD: u32 = 3;

/// A runtime-sized instruction loop. Each branch stores directly into the
/// register array so Enzyme can type the values, while reverse mode still has
/// to traverse dynamic control flow and retain its intermediate values.
#[autodiff_reverse(d_eval_tape, Const, Const, Const, Duplicated, Duplicated, Duplicated)]
#[inline(never)]
fn eval_tape(ops: &[u32], a: &[u32], b: &[u32], x: &[f64], regs: &mut [f64], out: &mut [f64]) {
    let n = ops.len();
    for i in 0..n {
        let ai = a[i] as usize;
        let bi = b[i] as usize;
        match ops[i] {
            OP_VAR => regs[i] = x[ai],
            OP_MUL => regs[i] = regs[ai] * regs[bi],
            OP_SIN => regs[i] = regs[ai].sin(),
            OP_ADD => regs[i] = regs[ai] + regs[bi],
            _ => unreachable!(),
        }
    }
    out[0] = regs[n - 1];
}

fn main() {
    // f(x0, x1) = x0*x1 + sin(x0), represented as runtime data.
    let ops = std::hint::black_box(vec![OP_VAR, OP_VAR, OP_MUL, OP_SIN, OP_ADD]);
    let a = std::hint::black_box(vec![0, 1, 0, 0, 2]);
    let b = std::hint::black_box(vec![0, 0, 1, 0, 3]);
    let x = std::hint::black_box(vec![1.25, 0.75]);
    let mut dx = vec![0.0; x.len()];
    let mut regs = vec![0.0; ops.len()];
    let mut dregs = vec![0.0; ops.len()];
    let mut out = [0.0];
    let mut dout = [1.0];

    d_eval_tape(
        &ops, &a, &b, &x, &mut dx, &mut regs, &mut dregs, &mut out, &mut dout,
    );

    let expected = [0.75 + 1.25_f64.cos(), 1.25];
    assert!((dx[0] - expected[0]).abs() < 1.0e-12);
    assert!((dx[1] - expected[1]).abs() < 1.0e-12);
    println!("primal={:.12}", out[0]);
    println!("gradient={dx:?}");
}

run-wasm.mjs:

import { readFile } from "node:fs/promises";
import { WASI } from "node:wasi";

const wasmPath = new URL(
  "./target/wasm32-wasip1/enzyme/enzyme-wasm-reverse-loop-mwe.wasm",
  import.meta.url,
);
const wasi = new WASI({ version: "preview1", args: [], env: {} });
const module = await WebAssembly.compile(await readFile(wasmPath));
const instance = await WebAssembly.instantiate(module, {
  wasi_snapshot_preview1: wasi.wasiImport,
});

try {
  wasi.start(instance);
} catch (error) {
  const stack = error instanceof Error ? error.stack ?? error.message : String(error);
  console.error(stack);
  if (stack.includes(".Lmalloc_bitcast_invalid")) {
    console.log("reproduced: Enzyme reverse-loop allocator call traps on wasm32");
    process.exit(0);
  }
  throw error;
}

throw new Error("unexpected success: the wasm32 reverse-mode call did not trap");

run.sh:

#!/usr/bin/env bash
set -euo pipefail

project_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
cd "$project_dir"

export RUSTFLAGS="${RUSTFLAGS:+$RUSTFLAGS }-Zautodiff=Enable"

echo "Building and running x86-64..."
cargo +nightly-2026-07-26 build --profile enzyme
./target/enzyme/enzyme-wasm-reverse-loop-mwe

echo
echo "Building and running wasm32-wasip1..."
cargo +nightly-2026-07-26 build --profile enzyme --target wasm32-wasip1
node run-wasm.mjs

x86-64 succeeds and checks the analytic gradient:
primal=1.886484619356
gradient=[1.0653223623952686, 1.25]

Instead, wasm32-wasip1 compiles successfully but traps during execution:

RuntimeError: unreachable
at .Lmalloc_bitcast_invalid
at diffe_...eval_tape
at d_eval_tape

The runner recognizes that exact failure and reports:
reproduced: Enzyme reverse-loop allocator call traps on wasm32

Meta

Using nightly-2026-07-26 (due to https://github.com/rust-lang/rust/issues/160470), -Zautodiff=Enable, and fat LTO

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running run.sh and compare the native and wasm32-wasip1 builds from main.rs. Inspect the generated d_eval_tape path around the .Lmalloc_bitcast_invalid trap, using run-wasm.mjs to reproduce it. Done means the wasm execution completes without that trap and produces the expected gradient shown in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, wasm
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.