shared-suspendable and shared-fixed as separate function types

During our discussion on https://github.com/WebAssembly/shared-everything-threads/issues/42, we discussed that a "safety valve" decision for JS function access, if we can't reach consensus on (strong/weak) thread-local functions, would be to (re)introduce a version of shared function that cannot have its execution suspended as part of a (hypothetical) shared continuation.

Currently our design doesn't permit nonshared parameters to shared functions in order to be forward-compatible with shared continuations, which would allow such a nonshared object to be smuggled into another thread by suspending execution and resuming in another thread. By forbidding such suspensions, we could allow nonshared parameters, and thus pass in a JS context (e.g. a struct containing unshared references to JS functions) as a regular parameter that would be threaded through execution (or a context local as sketched here), giving a mechanism to call JS functions from shared Wasm functions.

This issue is to discuss the design implications of this approach. A few initial points:

• As noted in my previous comment, the question of how the stack pointer is handled can be answered separately, as it is a scalar parameter. We could choose not to have thread-local globals (threading the stack pointer as a parameter), but still choose to support (strong/weak) thread-local functions.
• All calls to JS through the "context parameter" mechanism have to be indirect calls. Without a way to statically import JS functions as shared, unconditional inlining (the way V8 prefers) is likely infeasible.
  • Strong/weak thread-local functions just about make unconditional inlining feasible, so long as every instance-crossing is accompanied by a bitmap "is-on-fast-path" check. The feasibility of this bitmap approach is discussed/criticised here.
  • Given the above, maybe the only way to get unconditional inlining is to genuinely provide new JS builtins that are importable as shared, in the manner of string builtins.
• To wear my heart on my sleeve, I'd prefer just the shared-suspendable semantics with some mechanism for thread-local functions (weak if necessary) over the below. That being said, I don't think the below is awful, and it seems less controversial from a GC engineering perspective.

Note that, if we still believe that shared continuations will eventually exist, the below approach doesn't permanently solve our current problems, but instead pushes them into the future. Any compilation scheme wanting to use shared continuations will need to mark most functions as shared-suspendable, and so for such a scheme we'd still need to solve the same problem of JS access that we have in the current design (e.g. by introducing thread-local functions).

Design Sketch

Terminology

For the purposes of this discussion, I'm going to refer to functions as being either nonshared, shared-suspendable, or shared-fixed (a different name for shared-nonsuspendable, which is a mouthful). The distinction between these functions would be enforced by a static annotation on the function type.

nonshared functions are what we have today. Remember that in general, shared things can't capture nonshared things.

shared-suspendable functions are the "fully shared" functions we've been discussing before this point:

• No references, access, or calls to any module state with a nonshared type (globals/tables/functions)
• No nonshared parameters/locals
• No instructions that generate new nonshared references in the body (e.g. struct.new)
• These functions can be used to create shared continuations
  shared-fixed functions are somewhat more relaxed:
• Still no references, access, or calls to any module state with a nonshared type (globals/tables/functions)
• nonshared parameters/locals are allowed
• Instructions that generate new nonshared references in the body (e.g. struct.new) are allowed
• These functions cannot be part of a shared continuation, but can be part of a nonshared continuation

Intuitively, the call stack of a shared-suspendable function could be captured as part of a shared continuation, and resumed in another thread. This means it's not safe for the frame to ever capture a nonshared object, even transiently. In contrast, shared-fixed function calls are guaranteed to stay in the same thread for the entire duration of their execution. Therefore it's safe to pass in nonshared objects as parameters, and materialise them during the function call's execution. Prior to the standardisation of shared continuations, only shared-fixed functions would be definable.

Restrictions on calling

(EDIT: see this comment for an alternative approach with different restrictions)

shared-suspendable functions can always call shared-fixed functions with no restrictions. The extent to which shared-fixed functions can call shared-suspendable functions depends on some design decisions of stack switching. By default, all forms of shared-fixed->shared-suspendable call would be disallowed by validation (implying annotations/type tracking of [non]suspendable on relevant call instructions).

If the stack switching proposal includes a lexical barrier instruction (e.g. see here), it seems feasible to also include a concept of a "shared-only" barrier which traps upon an attempt to capture a shared continuation, but not a non-shared one. All forms of call to shared-suspendable functions could be allowed inside the body of this barrier. On reflection, I don't think that a shared-fixed call should implicitly introduce such a barrier, since this would mean that every shared-fixed call would have the implicit overhead of "Check if I'm in a continuation and if so, set the barrier bit". Instead I think the shared-only barrier should always be explicit (and would switch validation from shared-fixed mode to shared-suspendable mode within its body). @rossberg please correct me if I'm wrong about the above.

A shared-fixed function could also call a shared-suspendable function by wrapping the latter as a shared continuation and using a hypothetical resume_barrier instruction (as sketched here https://github.com/WebAssembly/stack-switching/issues/44#issuecomment-1909545807). If a "shared-only" barrier proves infeasible, this would be the only way to make a shared-fixed->shared-suspendable call.

Note that in either case, it's still ok for a shared-fixed function to hold a reference to a shared-suspendable function; only calling is complicated. This means that we don't need to distinguish between different kinds of shared for tables and globals - the [non]suspendable distinction is only needed for callable things.

Start by reading issues #42 and stack-switching #44, including the linked comments, then compare their assumptions with this design sketch. Done means reaching and documenting a consensus on whether separate shared-suspendable and shared-fixed function types should be adopted and what calling restrictions they require.