firebase / firebase/firebase-android-sdk
[Performance] High monitor lock contention in DefaultHeartBeatController and blockingRegister during cold startups on mid-range devices
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 710
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 34
Description
### [READ] Step 1: Are you in the right place?
Yes, this issue describes performance overhead, massive thread lock contention, and CPU core starvation originating directly from the `firebase-common` heartbeats storage and initialization path in this repository.
### [REQUIRED] Step 2: Describe your environment
- **Android Studio version:** `Android Studio Ladybug (2024.2.1)` (Profiling conducted on physical Moto G53 5G, Snapdragon 695, Android 14, API 34).
- **Firebase Component:** `firebase-common` (`DefaultHeartBeatController` & `JavaDataStorage` interactions affecting `firebase-installations`, `firebase-messaging`, `firebase-perf`, and `firebase-sessions`).
- **Component version:** `com.google.firebase:firebase-common:22.1.0` (as resolved under Firebase BoM `34.15.0`).
---
### [REQUIRED] Step 3: Describe the problem
While the deterministic deadlock reported in previous issues (such as nested `runBlocking` calls within DataStore edits) was successfully resolved in PR #8182 (`firebase-common:22.1.0`), **severe monitor lock contention still plagues the heartbeat storage path under high-concurrency cold startups (First Install scenario)**.
During early application startup, multiple modular Firebase components (FCM, Remote Config, Performance, Sessions) attempt to initialize and write/read metadata in parallel. On mid-range devices like the Moto G53 5G, this concurrent access to `DefaultHeartBeatController` triggers a massive thread synchronization storm.
Background thread lock queues of **up to 938 ms** are created. This heavy context-switching overhead saturates the CPU, forcing the OS scheduler to evict high-priority UI threads (`RenderThread`) to slower efficiency cores (Little Cores). This results in severe UI rendering delays, inflating EGLContext creation from a typical <5 ms to **220 ms**, and triggering severe frame jank (**588 ms** `Choreographer#doFrame`).
#### Steps to reproduce:
1. Integrate Firebase BoM `34.15.0` (with `firebase-common:22.1.0`) into a medium-to-large multi-module application using multiple Firebase SDKs (FCM, Remote Config, Performance, Sessions).
2. Perform a **First Install** run (cold start with no cached preferences or metadata) on a mid-range octa-core device (e.g., Moto G53 5G, Snapdragon 695 with 2x Gold cores + 6x Silver cores on Android 14).
3. Capture a system trace using **Perfetto** spanning the application initialization and first frame rendering.
4. Analyze the trace in the Perfetto UI or via `trace_processor_shell`. Observe massive concurrent `Thread::Init` slices, thread lock contentions, and `DefaultHeartBeatController` monitor bottlenecks on background threads.
#### Perfetto Trace Evidence (Extracted via `trace_processor`):
Across a batch of 16 Baseline Profile test runs, **100% of the traces** exhibited severe `firebase-common` monitor and thread lock contention, creating severe bottleneck queues:
```sql
-- Querying monitor contentions and thread lock bottlenecks on background threads:
SELECT name, ts, dur/1000000.0 as dur_ms
FROM slice
WHERE name LIKE '%monitor contention%' AND (name LIKE '%firebase%' OR name LIKE '%Firebase%')
-- Output Examples (Worst-Case Peak timings from RC29.3 benchmark batch):
1. Thread: "Firebase Background Thread #3" (TID 9955)
Slice: "monitor contention with owner Firebase Background Thread #3 at void com.google.firebase.heartbeatinfo.DefaultHeartBeatController..."
Duration: 938.67 ms (CRITICAL)
2. Thread: "Firebase-Messag" (TID 9933)
Slice: "Lock contention on a monitor lock (blocking from com.google.firebase.messaging.FirebaseMessaging.blockingRegister)"
Duration: 569.04 ms
3. Thread: "Firebase Background Thread #1" (TID 9915)
Slice: "monitor contention on DefaultHeartBeatController"
Duration: 754.75 ms
4. Thread: "Firebase Background Thread #0" (TID 9911)
Slice: "monitor contention on DefaultHeartBeatController"
Duration: 325.50 ms
```
#### Secondary Impact: CPU Core Starvation and Render Thread Degradation
Because of this parallel thread storm (including `Thread::Init` running 130+ times for newly spawned SharedPreferences and Firebase worker threads), the CPU Gold cores are fully choked.
CPU scheduling analysis (`sched` table) shows:
* The **`RenderThread`** is starved of big cores and is scheduled **68.9% of its active CPU time on Little Cores (CPUs 0-5)** (487.32 ms out of 706.79 ms).
* This starvation degrades native GPU driver initialization, causing `Creating EGLContext` on the RenderThread to take an astronomical **220.65 ms** (blocking the Main Thread's first layout `draw` for **259.99 ms**).
---
#### Relevant Code:
Even though PR #8182 eliminated the cyclic deadlock, the underlying design of `HeartBeatInfoStorage` and `DefaultHeartBeatController` relies heavily on global Java monitor locks (`synchronized` methods) wrapping asynchronous/blocking data access under high concurrency.
When 5-8 sub-modules of Firebase hit these methods in parallel on different background workers at the exact same millisecond on a cold start, they queue up linearly on `DefaultHeartBeatController`:
```java
// Sychronized methods in HeartBeatInfoStorage.java still trigger massive queues under high parallel load
synchronized void storeHeartBeat(long millis, String userAgentString) {
// Highly concurrent threads queue up here while background DataStore transactions are in-flight,
// causing up to ~938ms of monitor contention even if they eventually unlock without deadlocking.
...
}
```
Contributor guide
Research direction
Start by reading HeartBeatInfoStorage and DefaultHeartBeatController, especially the synchronized methods involved in heartbeat access, then inspect the blockingRegister path. Reproduce the cold-start scenario with Firebase BoM 34.15.0 and compare Perfetto traces for monitor contention and thread lock durations. The issue does not define a specific implementation or acceptance test, so the intended reduction in contention needs clarification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, java
- Domain
- mobile-dev, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100