hyperlight-dev / hyperlight-dev/hyperlight-wasm

Build can embed the wrong runtime after switching Pulley feature

Open
#545 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

lifecycle/needs review
Dominant language
Rust
Stars
728
Forks
39
Avg merge
2d 6h
Merged PRs (30d)
23

Description

Summary

Switching between native and Pulley builds can embed the wrong hyperlight-wasm-runtime guest. A native host build can retain the Pulley runtime and reject a valid x86 AOT component:

GuestError(GuestError, "Module was compiled for architecture 'x86_64'")

Reproduction

Use a consumer feature that enables hyperlight-wasm/pulley.

  1. Build and run with an x86 AOT component.
  2. Build and run with the Pulley feature and a Pulley AOT component.
  3. Build and run with the x86 AOT component again.

The third run can reject the x86 component. This reproduces with hyperlight-wasm 0.15.0 and Wasmtime 36.0.14.

Root cause

hyperlight-wasm/build.rs builds the embedded runtime into a shared nested target directory:

<target>/hyperlight-wasm-runtime/x86_64-hyperlight-none/<profile>/hyperlight-wasm-runtime

Native and Pulley Cargo units have separate OUT_DIR directories. Both generated wasm_runtime_resource.rs files use include_bytes! with this shared mutable path.

The nested Pulley build overwrites the runtime binary. Cargo can rebuild the native hyperlight-wasm unit because the include_bytes! input changed while its build script remains cached. The native unit then embeds the Pulley binary.

The shared output also permits a race between concurrent normal and Pulley builds.

Suggested fix

Copy the completed nested runtime into the outer build script's OUT_DIR. Point include_bytes! at that private copy. This preserves the shared nested compilation cache.

Concurrent builds also need one of these protections:

  • Hold a lock across the nested build and copy.
  • Use a separate nested target directory for each runtime configuration.

The configuration key must cover pulley, gdb, trace_guest, wasmtime_latest, and the WIT environment values. The outer OUT_DIR provides complete isolation but duplicates compilation work.

Regression coverage

Build a consumer in both sequences. Verify that the final component loads:

normal -> pulley -> normal
pulley -> normal -> pulley

Concurrent native and Pulley builds should produce correctly matched embedded runtimes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in hyperlight-wasm/build.rs and inspect how the nested runtime output and generated wasm_runtime_resource.rs use OUT_DIR and the shared target path. Reproduce the normal -> pulley -> normal and pulley -> normal -> pulley consumer sequences, then exercise concurrent native and Pulley builds. Done means each AOT component loads with its matching embedded runtime and concurrent builds remain correctly isolated.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, wasm
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.