Wasmtime C API: Possibility of switching to "abort" panic mode?

Hi!

(Please note that I don't know much about Rust, so please correct me if I'm saying anything that doesn't make sense.)

As far as I understand panics in Rust, they indicate that an unrecoverable/serious error has occured, after which the process should be terminated (to not run into undefined behavior).

In bytecodealliance/wasmtime-dotnet#192, we found that on **Windows**, when a panic occurs in the Wasmtime C API, it will raise an SEH Exception (e.g. with Win32's [`RaiseException`](https://learn.microsoft.com/en-us/windows/win32/api/errhandlingapi/nf-errhandlingapi-raiseexception) or [`_CxxThrowException`](https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/cxxthrowexception?view=msvc-170) from the MSVC runtime).

However, because .NET appears to use the same mechanism to handle exceptions, such a panic will surface as [`SEHException`](https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.sehexception) in a .NET application on the managed-to-native transition, which can be caught by the user if they have e.g. an `catch (Exception)` or `catch (SEHException)` clause. This means that in such a case, the process will not actually terminate, but will can continue to run, which could be problematic because we may now have undefined behavior with possible security implications.

This can even happen in `wasmtime-dotnet` if user code actually doesn't intend to catch `SEHException`:
When a .NET exception occurs in a wasm callback, `wasmtime-dotnet` will catch the exception (by using an `catch (Exception ex)` clause to catch all .NET exceptions), and transform it into a `wasm_trap_t*`, which is then returned at the native wasmtime callback. [^1]
Now, imagine there is a host-to-wasm and a wasm-to-host transition on the stack, and you call a wasmtime function that panics, resulting in a `SEHException` on Windows on the managed-to-native transition. Even if the user code on top of the wasm-to-host transition doesn't have a `catch (Exception)` or `catch (SEHException)` clause, the `SEHException` will be caught by [`Function`'s callback handler](https://github.com/bytecodealliance/wasmtime-dotnet/blob/972ef243be61ae60bf231fd22efd539456fbd819/src/Function.cs#L1959-L1962) and transformed into a `wasm_trap_t*`, which is then reported at the managed-to-native transition (like `wasmtime_call_func`) as `wasmtime_error_t*`, and that is thrown in .NET code as a `WasmtimeException`.
So even if you just use `catch (WasmtimeException ...)` e.g. around `Function.Invoke()` (as it is expected that such an exception may be thrown here), it can happen that you catch a Rust panic that was actually intended to terminate the process.

On Windows there is an alternative function [`RaiseFailFastException`](https://learn.microsoft.com/en-us/windows/win32/api/errhandlingapi/nf-errhandlingapi-raisefailfastexception), which bypasses all exception handlers and ensures the process is terminated (I assume this is also e.g. what .NET internally does in [`Environment.FailFast()`](https://learn.microsoft.com/en-us/dotnet/api/system.environment.failfast?view=net-6.0#system-environment-failfast(system-string))).
From Rust PR rust-lang/rust#32900, I understand that there is another panic mode (`abort`) which would have a similar effect to calling `RaiseFailFastException`.

Quoting @peterhuene from https://github.com/bytecodealliance/wasmtime-dotnet/pull/192#issuecomment-1341345790 (please see the conversation there for more background on this in `wasmtime-dotnet`):
> That said, I do think changing the panic mode for the C API to abort would make the most sense to address this issue. Would you mind opening an issue in the Wasmtime repo to explore doing so and reference this conversation?

Would it be possible to change the panic mode for the C API to `abort`, to ensure on Windows the process is terminated when a panic occurs?
(Note that e.g. on Linux, this is what already happens in .NET applications using `wasmtime-dotnet`, since the CLR cannot catch the panic there.)

Thank you!

[^1]: This ensures we don't let any .NET exception bubble through the native-to-managed transition. If `wasmtime-dotnet` wouldn't catch an exception thrown in .NET code, on Windows, the .NET CLR would unwind the stack up to the next .NET exception handler (that catches this exception type) even if there are managed-to-native and native-to-managed transitions on the stack, which seems to be incompatible with Wasmtime - see bytecodealliance/wasmtime-dotnet#187 for an example.

Start with Wasmtime's Rust build/configuration for the C API and its panic-handling path, then review the linked wasmtime-dotnet discussion for the Windows exception behavior. Compare the proposed abort mode across Windows and Linux, and confirm that the C API's documented behavior and relevant tests cover process termination when a panic occurs.

Written by the indexing model from the issue text.