WebAssembly / WebAssembly/WASI

[0.3.0-draft]: Move away from streams as an API?

Open
#757 24 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

P-cli
Dominant language
Rust
Stars
5.8k
Forks
333
Avg merge
2d 13h
Merged PRs (30d)
3

Description

I'd like to propose a possible radical restructuring of stdin/stdout/stderr and how they're modeled for 0.3.0. Specifically something like this for 0.3.0:

interface stdin {
  read: async func(amount: u32) -> result<list<u8>>;
}

interface stdout {
  write: func(data: list<u8>) -> result;
}

interface stderr {
  write: func(data: list<u8>) -> result;
}

Specifically this would do away with streams entirely and instead focus on just bytes. The stdin interface is tagged as async indicating that it will block waiting for input and bindings generators might want to do future-y things, but they can of course opt-out of that as well. For write and stdout/stderr they're not tagged as async and semantically they block the program while the write is happening.

This is naturally very different from both 0.2.0 and the current 0.3.0-draft, so I'll try to motivate why I'd propose this design instead:

  • At the OS level stdout/stderr are not async. AFAIK it's just not possible on Windows and on Unix no one does it as turning your end nonblocking automatically turns the other end nonblocking which is basically never expected nor what you want. That means that as an "abstraction over what platforms provide" pretending output is async is already a bit of a lie.
  • Printing to stdout/stderr is quite common but also low-level at the same time. Every language has some facility for printing to the screen and printing to stdout/stderr. No language I'm aware of exposes this as an async interface or as a stream. What I'm proposing here I feel better matches source languages where "just print the stuff before I keep going" is what's desired with these interfaces most of the time.
  • Historically when implementing 0.2.0 and what I'm forseeing with 0.3.0 interfaces is a lot of tricky questions. With 0.2.0 we went back-and-forth about what to do about stdout/stderr and how to implement them in Wasmtime. In the end we skipped the async part of streams entirely and the native implementations just block and are "always ready". I'll note stdin is a bit special but I don't think we can get away from that, so this is mostly 0.3.0. I see https://github.com/WebAssembly/wasi-cli/issues/64 as well for 0.3.0 and I also feel like it's asking a lot to have so much extra runtime support code just to write to stdout/stderr in language standard libraries. Overall I've always had the feeling that stdout/stderr management is causing a lot of headaches and integration questions when at the end of the day everyone typically wants something much simpler that's along the lines of "please just print this".

IIRC @pchickey and I basically concluded during 0.2.0 that we'd just come back to this at some point before 1.0.0 and rethink stdio. Personally I think now's a good as time as any as we transition from 0.2.0 to 0.3.0. There's of course downsides to the above such as "piping" is less obvious than before or redirection, but so far I'm not aware of any guest language which would support that. This could hypothetically be added in the future but I think it'd be best to start with a simple write and read function if we can.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no repository files or tests; it proposes replacing the 0.3.0-draft stdin, stdout, and stderr stream interfaces with byte-based functions. Start by reviewing the current 0.3.0 draft and wasi-cli issue 64, then trace how these interfaces are modeled and implemented. Done means the design is resolved and the relevant specification and implementations consistently reflect the decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
wasm
Domain
api, operating-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.