SIGABRT "Cannot transition thread [xxxx] from RUNNING with DONE_BLOCKING" in .NET `9.0.10` on Android.
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
### Description
We have an Android app that has been in production for over a year. Our last release was built against .Net `9.0.9` and was stable.
After releasing an update built against .Net `9.0.10`, we started noticing large amounts of app crashes. Our investigating showed the cause to be `SIGABRT "Cannot transition thread [xxxx] from RUNNING with DONE_BLOCKING"`.
___We also sometimes see "ASYNC_SUSPEND_REQUESTED with DONE_BLOCKING".___
Our tombstones had "/bin/ping" listed in "memory near". Testing revealed triggering `System.Net.NetworkInformation.Ping` at the same time the GC was running could cause the crash to occur.
We simplified our test to call `System.Diagnostics.Process.Start` while in parallel also calling `GC.Collect()`, which reproduced the crash (https://github.com/torutek/ThreadTransitionCrash). This a contrived example to reproduce the SIGABRT as fast as possible. This is not what our app is actually doing, we do not call `GC.Collect()` manually. We test connectivity to several devices on the network using `Ping`. When the GC and ping commands happen to line up, the crash occurs.
We tried doing a single ping while calling `GC.Collect()` in parallel, and it did not crash.
It looks like `Process.Start` (or `Ping`) is causing the issue when the GC is running at the same time. We believe the PR [Don't use vfork on android](https://github.com/dotnet/runtime/pull/118331) introduced or surfaced this issue.
__This is still broken on the current version of .Net 10.__
We have `AndroidEnableSGenConcurrent` disabled in production, but the issue occurs with it `enabled` as well.
### Reproduction Steps
We created a minimal Android app (https://github.com/torutek/ThreadTransitionCrash) that reproduces the issue consistently on device or emulator.
__The .Net SDK is set to `9.0.306` (.Net `9.0.10`) in the global.json file of the repro project.__
.Net SDK `9.0.306` must be installed.
Run sample android app:
1. Clone the repro project from the provided GitHub link.
2. Install Android workloads (from directory where the repro project is cloned):
1. `dotnet workload config --update-mode manifests`
2. `dotnet workload install ios tvos macos maccatalyst maui android`
3. `dotnet workload restore`
3. Build and run the project on an Android device or emulator.
4. Allow the app to run for several seconds (~20 seconds) until it crashes with the specified error message.
* On device seems to crash faster than on emulator.
5. Observe the crash logs to confirm the presence of the "Cannot transition thread [xxxx] from RUNNING with DONE_BLOCKING" message.
### Expected behavior
Runs for several minutes without crashing.
Reproduce steps to verify .Net `9.0.9` did not have the issue (from directory where the repro project is cloned):
1. Change `global.json` workloadVersion to sdk `9.0.305`
2. Change Android workloads (from directory where the repro project is cloned):
1. `dotnet workload config --update-mode manifests`
2. `dotnet workload install ios tvos macos maccatalyst maui android`
3. `dotnet workload restore`
3. Build and run the project on an Android device or emulator.
4. Should not crash (does not crash after 10 minutes).
### Actual behavior
Crashes after several seconds with `SIGABRT "Cannot transition thread [xxxx] from RUNNING with DONE_BLOCKING".`
Snapshot of tombstone:
```
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
Abort message: 'Cannot transition thread 0x7265a84c2cf0 from RUNNING with DONE_BLOCKING'
rax 0000000000000000 rbx 00007265a84c17c8 rcx 00007268bbeb8610 rdx 0000000000000006
r8 00007265b6496ee6 r9 00007265b6496ee6 r10 00007265a84c17d0 r11 0000000000000203
r12 000072664dbd9936 r13 00007265a84c2410 r14 000000000000638d r15 00000000000063b1
rdi 000000000000638d rsi 00000000000063b1
rbp 0000000000000001 rsp 00007265a84c17c0 rip 00007268bbeb8610
15 total frames
backtrace:
#00 pc 0000000000061610 /apex/com.android.runtime/lib64/bionic/libc.so (abort+192) (BuildId: fa337969c798946280caa45e2d71a2e7)
#01 pc 0000000000054ed7 /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonodroid.so (xamarin::android::Helpers::abort_application(_LogCategories, char const*, bool, std::__ndk1::source_location)+135) (BuildId: f56e6bd8a5a55c1f82faaa79e186f3b95942ac52)
#02 pc 0000000000020666 /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonodroid.so (xamarin::android::Helpers::abort_application(char const*, bool, std::__ndk1::source_location)+22) (BuildId: f56e6bd8a5a55c1f82faaa79e186f3b95942ac52)
#03 pc 000000000002f52a /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonodroid.so (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+138) (BuildId: f56e6bd8a5a55c1f82faaa79e186f3b95942ac52)
#04 pc 00000000001f598b /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonosgen-2.0.so (BuildId: b4845e3f94890e62328505fc1f38fd33cd5a9cc1)
#05 pc 00000000001f5a3d /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonosgen-2.0.so (BuildId: b4845e3f94890e62328505fc1f38fd33cd5a9cc1)
#06 pc 000000000020ffdc /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonosgen-2.0.so (BuildId: b4845e3f94890e62328505fc1f38fd33cd5a9cc1)
#07 pc 0000000000211371 /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonosgen-2.0.so (BuildId: b4845e3f94890e62328505fc1f38fd33cd5a9cc1)
#08 pc 00000000001c5910 /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonosgen-2.0.so (BuildId: b4845e3f94890e62328505fc1f38fd33cd5a9cc1)
#09 pc 00000000001b782c /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonosgen-2.0.so (BuildId: b4845e3f94890e62328505fc1f38fd33cd5a9cc1)
#10 pc 00000000001b5312 /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonosgen-2.0.so (BuildId: b4845e3f94890e62328505fc1f38fd33cd5a9cc1)
#11 pc 000000000027d706 /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonosgen-2.0.so (mono_runtime_invoke_checked+134) (BuildId: b4845e3f94890e62328505fc1f38fd33cd5a9cc1)
#12 pc 0000000000293b38 /data/app/~~AXLroOHJxVqz1V1etP9eug==/com.companyname.ThreadTransitionCrash-kUvY3gS6L-t5ToiXKGYHYg==/lib/x86_64/libmonosgen-2.0.so (BuildId: b4845e3f94890e62328505fc1f38fd33cd5a9cc1)
#13 pc 00000000000cd06a /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+58) (BuildId: fa337969c798946280caa45e2d71a2e7)
#14 pc 0000000000062d88 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+56) (BuildId: fa337969c798946280caa45e2d71a2e7)
```
### Regression?
Bug seems to have been introduced in .Net `9.0.10`. Possibly by [Don't use vfork on android](https://github.com/dotnet/runtime/pull/118331), we are unsure if it caused the issue, or exposed it.
### Known Workarounds
Revert to .Net `9.0.9` until the issue is resolved.
### Configuration
_No response_
### Other information
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.