The _other_ async

I'd love to have a Rust async runtime available for use within Hubris tasks. (Oxide wouldn't use it much because printing the state of an async-generated future from a debugger is a Hard Unsolved Problem, but, Oxide isn't everybody.)

This issue collects my notes on what such a thing would take.

Hey wait I thought you were Synchronous Guy

My Hubris announcement talk spent a lot of time on our synchronous interfaces and why I think they're good. I still feel that way, for the record.

The reason I like async is that it lets me write asynchronous code as if it were synchronous straight-line code (for the most part). This helps lower the cognitive overhead of explicit state machines. If you're going to write asynchronous code -- and we all wind up having to do so eventually, particularly when dealing with hardware or communications channels -- async is a lovely way to express it without all that asynchronous complexity waving in your face.

Dealing with the synchronous syscall interface

The elephant in the room is Hubris's synchronous syscall interface. I continue to feel that was the right design decision in general, but it presents a difficulty for tasks that want to use async: only one async task could be sending a message at any given time, and all the other async tasks within the Hubris task would block. (similarly for recv'ing messages.) This isn't necessarily an existential issue, however; there are applications, like managing a UART, where being able to have a couple of async tasks that don't use IPC at all would really simplify code.

This suggests a possible structure for such a program:

• There is a main event loop, like today. In the common case where a task acts as a server, it would recv messages/notifications and do stuff in response.
• One of the things it would do in response to messages/notifications is to poll the async runtime.
• async tasks within that runtime would communicate back out with the main loop using the normal sorts of mechanisms one uses for shared-memory inter-task communication -- message queues, semaphores, atomic integers in known places, etc.

After the stabilization of core::task in...what, late 2019? this is almost trivial to construct. It's essentially...

fn main() {
    let async_stuff = my_async_routine();

    // implementation of noop_waker omitted
    // see: https://github.com/cbiffle/lilist/blob/af16d6c194c3158d459d2928fe8edbaf7919918c/src/lib.rs#L804
    let waker = noop_waker();
    let mut ctx = core::task::Context::new(waker);

    loop {
        do_blocking_stuff();
        async_stuff.poll(&mut ctx)
}

async fn my_async_routine() -> ! {
    loop {
        futures::select! {
            _ => thing_one() => stuff;
            _ => thing_two() => other_stuff;
        }
    }
}

In a real version you'd want to have a more useful waker implementation, a way to communicate between async and blocking code, etc. But you get the idea.

async-compatible syscalls

Certain of Hubris's system APIs are amenable to multiplexing and could be used from async contexts:

• The timer API can be multiplexed by keeping track of all timer deadlines with e.g. a min-heap. This is no different from what the Hubris kernel itself is doing to multiplex the single hardware timer across tasks.
• Assuming panic = "abort", the sys_panic call gets the result you want whether called from async or normal code
• It's possible to use the borrow APIs to access a lease of an outstanding incoming message from async code, though you'd want interlocks to ensure that it doesn't try that after the (presumably synchronous) server code has replied
• What's more, it's also no big deal to reply to messages from async code. This plus borrows suggests that you could have a server that punts incoming messages (or only certain ones) into a blob of async code, which is responsible for replying.
• sys_irq_control, sys_refresh_task_id, sys_post are also fine.

I'm increasingly convinced that this set of APIs is sufficient to build an interesting/useful Hubris task using async internally.

async-incompatible syscalls and how to improve them

Other system API are not currently appropriate for use from async tasks within a real Hubris task:

• Tasks can still have only one sent message outstanding, so sys_send is not very useful to async code.
• Since we currently don't have a nonblocking form of sys_recv, it also probably doesn't do what you want in async code.
• sys_post works for sending notifications, but notifications go to tasks -- there is no way to notify an async sub-task.

There are ways one could change the system API to better support async use cases, if one wanted that. Some thoughts on that:

• The "async send" mechanism originally proposed for Hubris but never built would allow many async tasks within a "real" task to each have outgoing messages, by queueing them in a table in task RAM. (The kernel still refuses to participate in any queueing activities; the design would have been very similar to SENDA in MINIX.) However, the messages would be one-way in that design -- the async tasks could not receive replies.
• If async tasks want to recv messages, then other code needs some way to name them. Messages are sent to a Hubris task; there is currently no notion of any sub-entities within a task, nor is it immediately obvious to me how you'd implement such a thing. So this seems hard, and I'd be inclined to do recv and dispatch in the outer (synchronous) server loop as today.
• A similar problem applies to sys_post.

Start with Hubris's synchronous syscall interface and the issue's lists of async-compatible and async-incompatible syscalls, then review the proposed main event loop and core::task polling model. Done would require an agreed design and implementation scope for an async runtime and its syscall interactions; this issue currently provides notes rather than a concrete deliverable.