firebase / firebase/firebase-ios-sdk
Deadlock in -[APMIdentity appInstanceID]
- Dominant language
- C++
- Stars
- 6.7k
- Forks
- 1.8k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 75
Description
### Description
We've been investigating some strange behaviors in our app that were are hard to reproduce, but manifested when foregrounding the app and as we fired off requests to refresh our content (and also send analytics events).
Eventually we figured out it seemed like some kind of contention / deadlock issue.
Eventually we were able to catch an instance and attached the debugger, where we were able to see the stack trace. The root cause seems to be a deadlock caused by contention on the `dispatch_sync` call inside `-[APMIdentity appInstanceID]` as per this screenshot:
In our case we were using Swift concurrency which has has a number of concurrent system threads matching the number of physical cores on the system (in my case this was 6 with an iPhone 14 Pro) servicing Swift Tasks. Each thread of these 6 was blocked on the same `dispatch_sync` call and no forward progress could be made.
It would stand to reason that this issue could also occur in other situations as well where other threads are bounded, not just when using Swift concurrency.
My understanding is that using `dispatch_sync` is tricky to get right, and may lead to deadlocks when accessed recursively, especially when the caller's context is unknown. According to `man 3 dispatch_sync` this is conceptually implemented using a `dispatch_semaphore`, which is described as unsafe to use with Swift concurrency in WWDC 2021 "Swift concurrency: Behind the scenes":
Unfortunately it's hard to introspect the `GoogleAppMeasurement` package to determine what could be the cause of the deadlock, but we also are using AppCheck, Analytics, Auth, Firestore, Functions, Dynamic Links, and RemoteConfig, among others. I could imagine this could be some kind of case where other mutual exclusion techniques elsewhere in Firebase could trigger this scenario.
### Reproducing the issue
Reproducing this is tricky. I tried calling `Analytics.appInstanceID` in a tight loop concurrently (which was what was hanging) and wasn't able to reproduce. It seems likely that there's some other Firebase logic that has a dependency on `GoogleAppMeasurement#APMIdentity` that might be causing this issue. You can see that `FIRAppAttestProvider.state` is also blocked on something in that screenshot (unfortunately I don't have that stack available anymore), but they don't look related doing a basic inspection of `FIRAppAppAttestProvider` logic.
### Firebase SDK Version
10.10
### Xcode Version
14.3
### Installation Method
Swift Package Manager
### Firebase Product(s)
Analytics, App Check, Authentication, Crashlytics, Database, DynamicLinks, Firestore, Functions, In-App Messaging, Installations, Remote Config, Storage
### Targeted Platforms
iOS
### Relevant Log Output
_No response_
### If using Swift Package Manager, the project's Package.resolved
Expand Package.resolved snippet
```json
{
"pins" : [
{
"identity" : "abseil-cpp-binary",
"kind" : "remoteSourceControl",
"location" : "https://github.com/google/abseil-cpp-binary.git",
"state" : {
"revision" : "bfc0b6f81adc06ce5121eb23f628473638d67c5c",
"version" : "1.2022062300.0"
}
},
{
"identity" : "amplitude-ios",
"kind" : "remoteSourceControl",
"location" : "https://github.com/amplitude/Amplitude-iOS",
"state" : {
"revision" : "9e3081af2f3d45931b5404c4276c6c2e21ecd671",
"version" : "8.13.0"
}
},
{
"identity" : "analytics-connector-ios",
"kind" : "remoteSourceControl",
"location" : "https://github.com/amplitude/analytics-connector-ios.git",
"state" : {
"revision" : "b44981119e2efc0edd1d2e26831c3c4c0e0264e6",
"version" : "1.0.0"
}
},
{
"identity" : "appsflyerframework",
"kind" : "remoteSourceControl",
"location" : "https://github.com/AppsFlyerSDK/AppsFlyerFramework",
"state" : {
"revision" : "d1349c8fdd18a80a776298e4c5c3526e22feff78",
"version" : "6.8.1"
}
},
{
"identity" : "ats-sdk-ios",
"kind" : "remoteSourceControl",
"location" : "https://github.com/LiveRamp/ats-sdk-ios.git",
"state" : {
"revision" : "2983fdd06ab861f9286afb2ded727305441f6a67",
"version" : "1.4.0"
}
},
{
"identity" : "combine-schedulers",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/combine-schedulers",
"state" : {
"revision" : "aa3e575929f2bcc5bad012bd2575eae716cbcdf7",
"version" : "0.8.0"
}
},
{
"identity" : "firebase-ios-sdk",
"kind" : "remoteSourceControl",
"location" : "https://github.com/firebase/firebase-ios-sdk.git",
"state" : {
"revision" : "5034479ca0c4f32f299677fb0ba1fe4d3e3e20b4",
"version" : "10.10.0"
}
},
{
"identity" : "googleappmeasurement",
"kind" : "remoteSourceControl",
"location" : "https://github.com/google/GoogleAppMeasurement.git",
"state" : {
"revision" : "69f5b8425dd473d2f2475bba7f88f82cd9da0e36",
"version" : "10.10.0"
}
},
{
"identity" : "googledatatransport",
"kind" : "remoteSourceControl",
"location" : "https://github.com/google/GoogleDataTransport.git",
"state" : {
"revision" : "5056b15c5acbb90cd214fe4d6138bdf5a740e5a8",
"version" : "9.2.0"
}
},
{
"identity" : "googleutilities",
"kind" : "remoteSourceControl",
"location" : "https://github.com/google/GoogleUtilities.git",
"state" : {
"revision" : "871d43135925cde39ef7421d8723ce47edfdcc39",
"version" : "7.11.1"
}
},
{
"identity" : "grdb.swift",
"kind" : "remoteSourceControl",
"location" : "https://github.com/groue/GRDB.swift.git",
"state" : {
"revision" : "dd7e7f39e8e4d7a22d258d9809a882f914690b01",
"version" : "5.26.1"
}
},
{
"identity" : "grpc-binary",
"kind" : "remoteSourceControl",
"location" : "https://github.com/google/grpc-binary.git",
"state" : {
"revision" : "f1b366129d1125be7db83247e003fc333104b569",
"version" : "1.50.2"
}
},
{
"identity" : "gtm-session-fetcher",
"kind" : "remoteSourceControl",
"location" : "https://github.com/google/gtm-session-fetcher.git",
"state" : {
"revision" : "d415594121c9e8a4f9d79cecee0965cf35e74dbd",
"version" : "3.1.1"
}
},
{
"identity" : "gzipswift",
"kind" : "remoteSourceControl",
"location" : "https://github.com/1024jp/GzipSwift.git",
"state" : {
"revision" : "7a7f17761c76a932662ab77028a4329f67d645a4",
"version" : "5.2.0"
}
},
{
"identity" : "ios-sdk",
"kind" : "remoteSourceControl",
"location" : "https://github.com/spotify/ios-sdk",
"state" : {
"revision" : "c12c8630949492439e54b6482219c34bc7d65157",
"version" : "1.2.3"
}
},
{
"identity" : "leveldb",
"kind" : "remoteSourceControl",
"location" : "https://github.com/firebase/leveldb.git",
"state" : {
"revision" : "0706abcc6b0bd9cedfbb015ba840e4a780b5159b",
"version" : "1.22.2"
}
},
{
"identity" : "libwebp-xcode",
"kind" : "remoteSourceControl",
"location" : "https://github.com/SDWebImage/libwebp-Xcode.git",
"state" : {
"revision" : "0f3bdb28a1edc5e8e43876d3835d20c601ef331f",
"version" : "1.2.3"
}
},
{
"identity" : "lottie-ios",
"kind" : "remoteSourceControl",
"location" : "https://github.com/airbnb/lottie-ios",
"state" : {
"revision" : "314537ec697719fa33a9d48210f4d06a463286d3",
"version" : "3.4.3"
}
},
{
"identity" : "nanopb",
"kind" : "remoteSourceControl",
"location" : "https://github.com/firebase/nanopb.git",
"state" : {
"revision" : "819d0a2173aff699fb8c364b6fb906f7cdb1a692",
"version" : "2.30909.0"
}
},
{
"identity" : "nimbus-ios-sdk",
"kind" : "remoteSourceControl",
"location" : "https://github.com/timehop/nimbus-ios-sdk",
"state" : {
"revision" : "becc9351d48d51b8739843ca8a59a216e6c91edf",
"version" : "2.10.0"
}
},
{
"identity" : "onesignal-xcframework",
"kind" : "remoteSourceControl",
"location" : "https://github.com/OneSignal/OneSignal-XCFramework",
"state" : {
"revision" : "b87060d4a77ab6b5e96d6655b2587d3cb05e33e5",
"version" : "3.12.3"
}
},
{
"identity" : "persister",
"kind" : "remoteSourceControl",
"location" : "https://github.com/Lickability/Persister",
"state" : {
"branch" : "main",
"revision" : "556198feaa1b37cfb54328af43dcb2709002972a"
}
},
{
"identity" : "phonenumberkit",
"kind" : "remoteSourceControl",
"location" : "https://github.com/marmelroy/PhoneNumberKit",
"state" : {
"revision" : "9b051490589ae4f2fa861287407a43a87be7226c",
"version" : "3.5.8"
}
},
{
"identity" : "promises",
"kind" : "remoteSourceControl",
"location" : "https://github.com/google/promises.git",
"state" : {
"revision" : "3e4e743631e86c8c70dbc6efdc7beaa6e90fd3bb",
"version" : "2.1.1"
}
},
{
"identity" : "sdwebimage",
"kind" : "remoteSourceControl",
"location" : "https://github.com/SDWebImage/SDWebImage.git",
"state" : {
"revision" : "4178d12a44691182c2eb79c70281b14cb73f4cbf",
"version" : "5.14.3"
}
},
{
"identity" : "sdwebimageswiftui",
"kind" : "remoteSourceControl",
"location" : "https://github.com/SDWebImage/SDWebImageSwiftUI.git",
"state" : {
"revision" : "61fefe9c284fd41ddef77d02749e88f00c305196",
"version" : "2.2.2"
}
},
{
"identity" : "sdwebimagewebpcoder",
"kind" : "remoteSourceControl",
"location" : "https://github.com/SDWebImage/SDWebImageWebPCoder.git",
"state" : {
"revision" : "4416805045174938b11edb69a86e9f7dd0699952",
"version" : "0.9.1"
}
},
{
"identity" : "sentry-cocoa",
"kind" : "remoteSourceControl",
"location" : "https://github.com/getsentry/sentry-cocoa",
"state" : {
"revision" : "d277532e1c8af813981ba01f591b15bbdd735615",
"version" : "8.8.0"
}
},
{
"identity" : "swift-case-paths",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-case-paths",
"state" : {
"revision" : "870133b7b2387df136ad301ec67b2e864b51dda1",
"version" : "0.14.0"
}
},
{
"identity" : "swift-clocks",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-clocks",
"state" : {
"revision" : "20b25ca0dd88ebfb9111ec937814ddc5a8880172",
"version" : "0.2.0"
}
},
{
"identity" : "swift-collections",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-collections",
"state" : {
"revision" : "f504716c27d2e5d4144fa4794b12129301d17729",
"version" : "1.0.3"
}
},
{
"identity" : "swift-composable-architecture",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-composable-architecture.git",
"state" : {
"revision" : "3e8eee1efe99d06e99426d421733b858b332186b",
"version" : "0.52.0"
}
},
{
"identity" : "swift-custom-dump",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-custom-dump",
"state" : {
"revision" : "de8ba65649e7ee317b9daf27dd5eebf34bd4be57",
"version" : "0.9.1"
}
},
{
"identity" : "swift-dependencies",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-dependencies",
"state" : {
"revision" : "6bb1034e8a1bfbf46dfb766b6c09b7b17e1cba10",
"version" : "0.2.0"
}
},
{
"identity" : "swift-gen",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-gen.git",
"state" : {
"revision" : "5bd20fb662e1ead7ee47df6bb0a15398295f2e06",
"version" : "0.4.0"
}
},
{
"identity" : "swift-identified-collections",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-identified-collections",
"state" : {
"revision" : "ad3932d28c2e0a009a0167089619526709ef6497",
"version" : "0.7.0"
}
},
{
"identity" : "swift-nonempty",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-nonempty.git",
"state" : {
"revision" : "8074e5f2e4a6de5a42476e545cc33438021aa2a4",
"version" : "0.4.0"
}
},
{
"identity" : "swift-overture",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-overture",
"state" : {
"revision" : "7977acd7597f413717058acc1e080731249a1d7e",
"version" : "0.5.0"
}
},
{
"identity" : "swift-package-manager-google-mobile-ads",
"kind" : "remoteSourceControl",
"location" : "https://github.com/googleads/swift-package-manager-google-mobile-ads.git",
"state" : {
"revision" : "3a03edf8728c50bc380b4a9187adf178ea477a6d",
"version" : "9.12.0"
}
},
{
"identity" : "swift-package-manager-google-user-messaging-platform",
"kind" : "remoteSourceControl",
"location" : "https://github.com/googleads/swift-package-manager-google-user-messaging-platform.git",
"state" : {
"revision" : "3b924ce3313a5fd2fc6a8dc889a8c38f76890fe3",
"version" : "2.0.0"
}
},
{
"identity" : "swift-protobuf",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-protobuf.git",
"state" : {
"revision" : "e1499bc69b9040b29184f7f2996f7bab467c1639",
"version" : "1.19.0"
}
},
{
"identity" : "swift-tagged",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-tagged",
"state" : {
"revision" : "f3f773a5e13f3c8f0ab1ce2ae6378058acefa87d",
"version" : "0.7.0"
}
},
{
"identity" : "swiftui-navigation",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swiftui-navigation",
"state" : {
"revision" : "0a0e1b321d70ee6a464ecfe6b0136d9eff77ebfc",
"version" : "0.7.0"
}
},
{
"identity" : "vungleadssdk-swiftpackagemanager",
"kind" : "remoteSourceControl",
"location" : "https://github.com/Vungle/VungleAdsSDK-SwiftPackageManager.git",
"state" : {
"revision" : "3ffeb9316a625e9486f850da15dff3713dd6f9b5",
"version" : "7.0.1"
}
},
{
"identity" : "xctest-dynamic-overlay",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/xctest-dynamic-overlay.git",
"state" : {
"revision" : "62041e6016a30f56952f5d7d3f12a3fd7029e1cd",
"version" : "0.8.3"
}
}
],
"version" : 2
}
```
### If using CocoaPods, the project's Podfile.lock
Expand Podfile.lock snippet
```yml
Replace this line with the contents of your Podfile.lock!
```
Contributor guide
Assessment
This issue has not been assessed yet.