WebAssembly / WebAssembly/tool-conventions
Document expected import/export behavior for tools
Nobody has claimed this yet.
- Dominant language
- WebAssembly
- Stars
- 372
- Forks
- 75
- PR merge metrics
- No merged PRs in 30d
Description
Today, my compiler successfully emitted a working wasm file with the help of the LLVM back-end and the wasm-ld linker. Figuring out how to do so proved to be a frustrating and time-consuming challenge in large part due to breaking LLVM changes and a lot of undocumented behavior around how the tools handle import and export sections. Finding helpful answers required gathering together stray data cast around in obscure websites (and running across a lot of advice that is now wrong) and having to read in detail the LLVM source code used to generate and link wasm modules.
So ... I recommend that this repo contain a document on expected import/export behavior across tools. Some of this information should also be selectively disseminated elsewhere, where appropriate. Here are some information I believe would be useful to cover:
- How LLVM (both the backend & linker) can decide by default what names to identify as exported and what to be imported. At some point between LLVM v5 and v7, the LLVM backend was neutered to be unable to generate a usable wasm file on its own: it generates no export section and hardcodes only two imports (for memory and table) with loader-unusable names. Not only is this generated wasm unusable, it forces the developer to manually enumerate what to export/import as part of the linker step, a mistake-prone process not required when creating object files and executables for native OS. In addition, the linker strips off the memory/table import and assumes they are to be exported instead.
Why would it not make more sense for wasm files generated by a compiler to be workable as is (without requiring a linker step) and let the compiled programs specify what to import/export based on the language's visibility attribute (a concept baked into LLVM) or alternatively based on the DLLimport/export (a concept also baked into LLVM)? If the linker is given wasm files with existing import/export sections, it need not strip or override them, but rather merge them. If no import/export info exists, or it needs to be overridden, the linker's many options currently supported can be used to add or change these settings. But it feels like the linker has less information available to make default decisions than the compiler does, so let the compiler lead if it chooses to.
-
Clearly document the linker's import/export manipulation options somewhere and describe explicitly what they do: --export, --export-dynamic, --export-table, --import-table, --export-memory, --import-memory, ---allow-undefined-file (!).
-
Establish a consistent default behavior re: --import-memory vs. --export-memory. Personally, I would select --import-memory as the default because it offers more flexibility in sharing and sizing memory across wasm modules. But whichever is chosen, there should be an agreement on the module name ('env'?) and the name of memory ('memory' vs. '__linear_memory'). Ditto for the table, which will play a larger role evidently in dynamic loading of wasm modules.
Note as well that the LLVM backend generates text-based wat files whose import/export sections don't match what is generated in the binary wasm file, making diagnosing problems harder because you don't expect to have to use the wabt tools to see what was really generated.
The Javascript documentation describing the instantiation also should be beefed up, to describe clearly how differently to handle when memory is imported vs. exported (and what that means), and that when imported, the import module should be named 'env' and the memory is expected to be called 'memory'. Ditto for the table's conventions.
I have no experience with the enscripten toolchain and backward compatibility issues, which no doubt complicate these decisions. I suspect I have gotten some stuff wrong (sorry). My intent here is to help make it easier for those that will come along afterwards. Perhaps other compilers (e.g., Rust or Zig, e.g., https://github.com/ziglang/zig/issues/1570) might also have valuable feedback on these standards before what people do solidifies too much more, making it impossible to corral in.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by surveying the LLVM backend, wasm-ld linker options, WAT output, and JavaScript instantiation guidance named in the issue. Resolve the proposed import/export conventions and compatibility questions before drafting documentation. Done means the expected defaults, option behavior, naming conventions, and imported-versus-exported memory and table handling are documented consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, wasm
- Domain
- compilers, documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100