emscripten-core / emscripten-core/emscripten

Syscalls vs. user calls in emscripten's main thread call dispatch system

Open
#14,292 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
27.6k
Forks
3.6k
Avg merge
1d 1h
Merged PRs (30d)
105

Description

The emscripten cross-thread call dispatch system (i.e. the one used by `emscripten_async_run_in_main_thread`, `emscripten_dispatch_to_thread` etc as well as syscalls automatically proxied to the main thread) has historically mostly just been used for proxied syscalls IIUC. But has recently started to be used for user code too, because, that's a useful feature to have.

Currently emscripten processes the queued calls in several places:
1) on a request via `postMessage`from another thread (this is sort of the "expected" place, and what happens when the receiving thread is idle)
1) Before (and/or after) most places that the thread runtime could sleep:
1) during `emscripten_thread_sleep()` before and after its call to `emscripten_futex_wait()` (which implements the actual sleep)
1) during `pthread_join` before its futex wait
1) in `__timedwait()` and `pthread_barrier_wait` on the main thread before or after its futex wait (and also in `__wait()` but only when cancellation is async... why this mismatch?)
2) *inside* `emscripten_futex_wait` itself, in the busy loop (on the main thread)

My understanding from looking at the code is that all these extra queue processing events are primarily to ensure that syscalls are handled in a timely manner (since it means that they will get run before the event is delivered on the main thread's event queue).

The queue does provide the guarantee that it will not re-enter (e.g. if the user callback calls `malloc` which takes a lock, another event will not be dispatched).
However one user (who is implementing a UI event dispatch system on top of the emscripten queue) ran into a deadlock when they dispatched one of their events manually from JS, then their callback called malloc, which took the lock and dispatched another event. This is arguably a bug in their event dispatch system, but I can see how it would be useful to provide the guarantee that emscripten-queue calls will only be dispatched "directly" from the web event queue, rather than when user code is already on the stack.

So I have a couple of general questions:
1) Is my understanding of why we have these extra dispatches correct? Presumably @juj knows?
2) Is emscripten's internal use of the message queue only for proxied syscalls, or are there others? Are there any cases where those can call user code, or can this issue only happen when users enqueue their own calls?
3) If we add a mechanism to fix this problem (presumably it would be either a separate queue for syscalls vs user calls, or perhaps a second "class" of events that would only be dispatched from e.g. the postMessage handler), would there be any other use cases that it should address?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.