bytecodealliance / bytecodealliance/wasmtime

Support mmap flag MAP_JIT for MACOS

Open
#11,989 14 comments 0 reactions 0 assignees View on GitHub
wasmtime:platform-support
Dominant language
Rust
Stars
18.6k
Forks
1.8k
Avg merge
1d 18h
Merged PRs (30d)
126

Description

Hello,

During the development of our application, we’ve introduced WebAssembly (WASM) as a plugin system. Our software is a security terminal, and we’re now encountering macOS memory permission-related issues.

We refer to Apple’s official documentation:
https://developer.apple.com/documentation/apple-silicon/porting-just-in-time-compilers-to-apple-silicon#Enable-the-JIT-entitlements-for-the-Hardened-Runtime

In short, we need to allocate executable JIT memory using:
```c
mmap(NULL, wasm_plugin.size(), PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANON | MAP_JIT, -1, 0)
```
and then write into it using the macOS-specific API `pthread_jit_write_with_callback_np`. Both `MAP_JIT` and `pthread_jit_write_with_callback_np` are macOS-specific APIs.

For example, we implemented a function like this:

```cpp
struct jit_code {
void* jit_region;
void *dst;
size_t instructions_length;
};

int jit_writing_callback(void *context) {
struct jit_code *code = (struct jit_code *)context;
memcpy(code->dst, code->jit_region, code->instructions_length);
return 0;
}
void* mmap_from_vector(const std::vector& plugin_content) {
void* jit_mem = mmap(NULL, plugin_content.size(),
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANON | MAP_JIT,
-1, 0);
if (jit_mem == MAP_FAILED) {
// "jit_mem mmap failed"
return MAP_FAILED;
}

struct jit_code jtcode;
jtcode.jit_region = (void*)plugin_content.data();
jtcode.dst = jit_mem;
jtcode.instructions_length = plugin_content.size();
if (pthread_jit_write_with_callback_np(jit_writing_callback, &jtcode) != 0) {
munmap(jit_mem, plugin_content.size());
return MAP_FAILED;
}
return jit_mem;
}
```

The allocated `jit_mem` is thus marked as executable JIT memory.

At the time we began development, Wasmtime does not provide native C/C++ APIs, so our approach is to build a dynamic library in Rust using Wasmtime and export C-compatible functions for WASM loading, destruction, and function invocation, which our application then calls.

However, after allocating memory using the above method and invoking the exported WASM functions through these C interfaces, the program crashes with:
```
Exception Type: EXC_BAD_ACCESS (SIGKILL - Code Signature Invalid)
```
This indicates that the memory permissions are still invalid at runtime.

We quickly realized the root cause: when we call `Component::new` and pass in the WASM binary, Wasmtime internally allocates **new memory regions** during instance creation—meaning the memory we pre-allocated at the upper layer is **not actually used** for code execution.

After reviewing Wasmtime's source code, we found system-level `mmap` wrappers in:
`crates/wasmtime/src/runtime/vm/sys/unix/mmap.rs`

Our questions are:

1. When Wasmtime allocates `CodeMemory` on macOS, does it use the `mmap` implementation in this file?
2. Is there any way to extend this implementation to support macOS’s special memory allocation requirements (i.e., using `MAP_JIT` and writing via `pthread_jit_write_with_callback_np`)?
3. If I modify this code directly, will it affect memory allocations other than CodeMemory? Because, as expected, only executable code needs to be allocated in this special way.

Thank you!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.