dotnet / dotnet/android

NDK crashes symbolification

Open
#8,832 13 comments 0 reactions 2 assignees Claimed by @jonathanpeppers View on GitHub
Area: App+Library Build enhancement
Dominant language
C#
Stars
2.1k
Forks
579
Avg merge
1d 20h
Merged PRs (30d)
257

Description

### Android application type

.NET Android (net7.0-android, net8.0-android, etc.)

### Affected platform version

.NET 8.0.200, workload 34.0.52/8.0.100

### Description

Since updating from Xamarin to .NET 8 we are getting a lot of native crashes from production users. In order to investigate the reason, I'm trying to find a way to symbolificate them.

Our crash monitoring system is based on AppCenter, which uses Google Breakpad internally, but at this point I would be happy with any manual way of symbolification, either with Breakpad's [`minidump_stackwalk`](https://github.com/microsoft/appcenter-sdk-android-breakpad/blob/master/src/processor/minidump_stackwalk.cc), [`ndk-stack`](https://developer.android.com/ndk/guides/ndk-stack), or anything else.

First, I noticed that by default there are no debug symbols at all in the `.dll.so` files produced by AOT under . Running `readelf -Ws` usually gives something like that:

```
Symbol table '.dynsym' contains 2 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000003b30 0 OBJECT GLOBAL DEFAULT 9 mono_aot_file_info
```

I have found that setting `_AndroidAotStripLibraries` to `false` disables library stripping, but judging by the underscore in its name it's private API:

https://github.com/xamarin/xamarin-android/blob/51bcefdbf08a2c17ab0633d5aee37f22acf5164d/src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.Aot.targets#L93

Now `readelf` shows that there are debug symbols inside:

```
Symbol table '.dynsym' contains 2 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000003b30 0 OBJECT GLOBAL DEFAULT 9 mono_aot_file_info

Symbol table '.symtab' contains 40 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 00000000000014e0 0 NOTYPE LOCAL DEFAULT 6 jit_code_start
2: 00000000000014f0 83 FUNC LOCAL DEFAULT 6 AndroidSymbols_MainActivity_OnCreate_Android_OS_Bundle
3: 0000000000003d78 0 OBJECT LOCAL DEFAULT 10 mono_aot_AndroidSymbols_got
4: 0000000000001550 36 FUNC LOCAL DEFAULT 6 AndroidSymbols_MainActivity_NativeCrash
5: 0000000000001580 42 FUNC LOCAL DEFAULT 6 AndroidSymbols_MainActivity__ctor
...
```

However, if I crash the app and try to symbolificate the dump using the `.dll.so` with symbols e.g. with `ndk-stack`, it doesn't map the addresses to the symbols:

```
********** Crash dump: **********
Build fingerprint: 'Android/sdk_gphone_x86_64/generic_x86_64:8.0.0/OSR1.180418.026/6741039:userdebug/dev-keys'
#00 0x00000000000212c1 /system/lib64/libc.so (strlen+17)
#01 0x0000000000019582 <-- I would expect to see AndroidSymbols_MainActivity_NativeCrash here
#02 0x00000000000000d2
#03 0x0000000000000007
Crash dump is completed
```

I have also tried disabling assembly compression with `false`, but it didn't change anything in terms of symbolification.

I am assuming I am still not building the app correctly for symbolification, so I would appreciate any help with that.

### Steps to Reproduce

I have a repro project here, that crashes with an NDK crash on launch (run in Release): https://github.com/akravch/AndroidSymbols. `_AndroidAotStripLibraries` is set to `false`.

Also attaching [`readelf` output](https://github.com/xamarin/xamarin-android/files/14747282/readelf.txt) for the unstripped `.dll.so`, [`ndk-stack` output](https://github.com/xamarin/xamarin-android/files/14747281/ndk-stack.txt) and the [`.dll.so`](https://github.com/xamarin/xamarin-android/files/14747295/AndroidSymbols.dll.zip) itself.

### Did you find any workaround?

No.

### Relevant log output

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.