bytecodealliance / bytecodealliance/wasm-tools

wit-parser: merging interfaces is too restrictive

Open
#1,191 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.8k
Forks
351
Avg merge
1d 6h
Merged PRs (30d)
46

Description

I'm attempting to publish `wasi:sockets` to the preview registry.

The `wasi:sockets` package depends on binary packages `wasi:io` and `wasi:poll`; `wasi:io` also has a transitive dependency on `wasi:poll`.

The definitions of these packages are coming from binary packages downloaded from the registry and are guaranteed to be transitively compatible in their definitions.

However, when I try to build the `wasi:sockets` package, sometimes it succeeds:

```
$ RUST_LOG=debug wit build
DEBUG wit::commands::build > executing build command
INFO wit::lock > opening lock file `/Users/peterhuene/src/peterhuene/wasi/sockets/wit.lock`
INFO cargo_component_core::lock > dependency package `wasi:io` from registry `default` with requirement `^0.2.0-rc` was resolved by the lock file to version 0.2.0-rc+20230901
INFO cargo_component_core::lock > dependency package `wasi:poll` from registry `default` with requirement `^0.2.0-rc` was resolved by the lock file to version 0.2.0-rc+20230901
INFO cargo_component_core::registry > version 0.2.0-rc+20230901 of registry package `wasi:io` from registry `default` is already in client storage
INFO cargo_component_core::registry > version 0.2.0-rc+20230901 of registry package `wasi:poll` from registry `default` is already in client storage
DEBUG wit_component::decoding > decoding a WIT package encoded as wasm
DEBUG wit_component::decoding > type export for `pollable` is a new type
DEBUG wit_component::decoding > decoding a WIT package encoded as wasm
DEBUG wit_component::decoding > decoding import instance export `pollable`
DEBUG wit_component::decoding > type export for `pollable` is a new type
DEBUG wit_component::decoding > type export for `pollable` is an alias
DEBUG wit_component::decoding > type export for `stream-status` is a new type
DEBUG wit_component::decoding > type export for `input-stream` is a new type
DEBUG wit_component::decoding > type export for `output-stream` is a new type
DEBUG wit_parser::ast::resolve > toposort for interfaces and worlds in order: ["network", "instance-network", "ip-name-lookup", "tcp", "tcp-create-socket", "udp", "udp-create-socket"]
Created package `sockets.wasm`
```

and sometimes it fails:

```
$ RUST_LOG=debug wit build
DEBUG wit::commands::build > executing build command
INFO wit::lock > opening lock file `/Users/peterhuene/src/peterhuene/wasi/sockets/wit.lock`
INFO cargo_component_core::lock > dependency package `wasi:io` from registry `default` with requirement `^0.2.0-rc` was resolved by the lock file to version 0.2.0-rc+20230901
INFO cargo_component_core::lock > dependency package `wasi:poll` from registry `default` with requirement `^0.2.0-rc` was resolved by the lock file to version 0.2.0-rc+20230901
INFO cargo_component_core::registry > version 0.2.0-rc+20230901 of registry package `wasi:io` from registry `default` is already in client storage
INFO cargo_component_core::registry > version 0.2.0-rc+20230901 of registry package `wasi:poll` from registry `default` is already in client storage
DEBUG wit_component::decoding > decoding a WIT package encoded as wasm
DEBUG wit_component::decoding > decoding import instance export `pollable`
DEBUG wit_component::decoding > type export for `pollable` is a new type
DEBUG wit_component::decoding > type export for `pollable` is an alias
DEBUG wit_component::decoding > type export for `stream-status` is a new type
DEBUG wit_component::decoding > type export for `input-stream` is a new type
DEBUG wit_component::decoding > type export for `output-stream` is a new type
DEBUG wit_component::decoding > decoding a WIT package encoded as wasm
DEBUG wit_component::decoding > type export for `pollable` is a new type
DEBUG wit_parser::ast::resolve > toposort for interfaces and worlds in order: ["network", "instance-network", "ip-name-lookup", "tcp", "tcp-create-socket", "udp", "udp-create-socket"]
error: failed to merge world of dependency `wasi:poll`

Caused by:
0: failed to merge package `wasi:poll` into existing copy
1: failed to merge interface `poll`
2: expected function `drop-pollable` to be present
```

The unstable order is coming from the `wit` tool, but that's not really the issue here.

The issue is that `Resolve::merge` requires interfaces to be identical in their types and functions, but this is too restrictive when dealing with binary-encoded packages that may have a subset of transitive interface definitions (i.e. used types).

In this case, one `wasi:poll` has the full package definition (the direct `wasi:poll` dependency from `wasi:sockets`) and the other just the types from `wasi:poll` used in `wasi:io`.

Despite the comments in `MergeMap::build_interface`, the merge should really be the union of the two with any overlap being required to be identical.

If we keep it how it currently works, then an encoded WIT package _must_ encode the entire package definition of dependencies and not just what is used from those packages; without doing so, there's no way to faithfully "reconstitute" the dependency definitions.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.