agittins / agittins/bermuda

Performance: cProfile analysis shows ~12% CPU consumption — optimization opportunities for calculate_data, list.insert, and scanner stamp lookups

Aperta
#738 0 commenti 1 reazione 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
2k
Fork
71
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Summary

I ran a 60-second `profiler.start` cProfile session on my Home Assistant instance and found that **Bermuda is the single largest CPU consumer** among all integrations, accounting for ~7.2s of CPU self-time (~12% of the 60s profiling window) across its hot paths. I'm sharing the full analysis here in the spirit of constructive feedback — Bermuda is a fantastic integration and I'd love to help improve its performance.

## Environment

- **Home Assistant Core**: 2026.2.3
- **Home Assistant OS**: 17.1
- **Hardware**: Home Assistant Green (~4 GB RAM)
- **Bermuda**: installed via HACS (latest)
- **BLE Proxies**: 6 ESPHome devices
- **Bluetooth devices seen**: ~258 devices, ~360 advertisements per cycle
- **Update interval**: ~1 second (60 cycles observed in 60s)

## Profiling Methodology

Used the built-in HA `profiler` integration:

```yaml
action: profiler.start
data:
seconds: 60
```

Output files:
- `profile.1772397475817944.cprof`
- `callgrind.out.1772397475817944`

Profiling was done during normal operation, no special load was applied.

## Profile Results

**Profile duration**: 59.9 seconds | **Total calls**: 2,725,309 | **Unique functions**: 3,681

### Top Bermuda-Specific Bottlenecks

| Rank | Function | Calls | Self Time | Cum. Time | Per-Call |
|------|----------|-------|-----------|-----------|---------|
| 1 | `bermuda_advert.calculate_data` | 46,534 | 1.174s | 1.380s | 25µs |
| 2 | `bermuda_advert.update_advertisement` | 21,584 | 0.456s | 1.278s | 59µs |
| 3 | `bermuda_device.calculate_data` | 15,462 | 0.409s | 1.913s | 124µs |
| 4 | `coordinator._async_gather_advert_data` | 60 | 0.399s | 2.368s | 39ms |
| 5 | `bermuda_device.async_as_scanner_get_stamp` | 53,296 | 0.279s | 0.440s | 8µs |
| 6 | `bermuda_device.process_advertisement` | 21,584 | 0.249s | 1.553s | 72µs |
| 7 | `bermuda_device.process_manufacturer_data` | 1,296 | 0.114s | 0.230s | 178µs |
| 8 | `bermuda_advert._update_raw_distance` | 12,167 | 0.096s | 0.118s | 10µs |
| 9 | `coordinator._get_or_create_device` | 21,644 | 0.086s | 0.086s | 4µs |
| 10 | `coordinator._async_update_data_internal` | 60 | 0.083s | 4.229s | 70ms |

**Total Bermuda CPU self-time**: ~7.2s out of 60s profiling window

### External / Network Bottlenecks (non-Bermuda, for context)

| Component | Calls | Cum. Time | Notes |
|-----------|-------|-----------|-------|
| PlayStation Network API | 10 | 3.02s | HTTP calls to PSN auth |
| HTTP requests (generic) | 46 | 3.89s | Dominated by socket read waits |
| snitun multiplexer | 3,288 | 1.65s | Nabu Casa remote access tunnel |

### High Call-Count Functions Worth Noting

| Function | Calls in 60s | Notes |
|----------|-------------|-------|
| `is_remote_scanner` | 53,717 | Property check — confirm no extra logic |
| `async_as_scanner_get_stamp` | 53,296 | Potential for per-cycle caching |
| `len()` | 316,399 | General Python overhead |
| `list.insert()` | 105,596 | **O(n) front-insertion — potential `deque` candidate** |

## Key Observations

1. **`bermuda_advert.calculate_data` is the single hottest function** — called 46,534 times in 60s (~775/sec). Since it's called from `bermuda_device.calculate_data` (15,462 times), the inner loop multiplier is roughly 3x adverts per device per cycle.

2. **`_async_gather_advert_data` is the heaviest per-call operation** — at 39ms cumulative per call (60 calls total = 2.37s), this is the main data-gathering loop that fans out into device/advert `calculate_data`.

3. **`list.insert()` is called 105,596 times** — inserting into lists (especially at the front) is O(n). If these are history/stamp sliding-window buffers, switching to `collections.deque(maxlen=N)` would reduce this to O(1).

4. **`async_as_scanner_get_stamp` is called 53,296 times** — this suggests redundant lookups. Caching the stamp per-scanner per-cycle could eliminate most of these calls.

5. **Update cycle timing**: 60 cycles x 70ms average = 4.2s of Bermuda processing per minute. This is within acceptable limits currently, but leaves limited headroom as device count grows (I have ~258 devices visible).

## Suggested Optimizations

### High Impact
- **Cache or memoize `calculate_data` results** for adverts that haven't changed since the last cycle. 46K calls per minute is aggressive if many adverts are stale.
- **Batch `_async_gather_advert_data`** — the profile shows it processes sequentially. Consider grouping by scanner or area to reduce redundant iterations.
- **Replace `list.insert()` with `collections.deque(maxlen=N)`** for any sliding-window history buffers (105K inserts/min suggests front-insertion into lists).

### Medium Impact
- **Reduce `async_as_scanner_get_stamp` call frequency** — 53K calls suggests redundant lookups. Cache the stamp per-scanner per-cycle.
- **Skip `calculate_data` for unchanged adverts** — if the RSSI/timestamp hasn't changed since last cycle, the distance calculation result is identical.

### Low Impact / Hygiene
- **`is_remote_scanner` property** — called 53K times, likely a simple attribute check but worth confirming it doesn't trigger extra logic.
- **`count_active_devices`** — runs every cycle (60 calls, 19ms). If device count is stable between cycles, cache the result.

## Resource Impact on HA Green

On the resource-constrained HA Green (4-core ARM, 4GB RAM), this translates to:
- Bermuda Core CPU: contributes significantly to the overall 5.14% HA Core CPU usage
- RAM: 6,904 State objects in memory (across all integrations), with Bermuda contributing device_tracker + sensor entities for tracked devices
- The 1-second update interval means Bermuda's 70ms per cycle consumes ~7% of each second's available processing time on a single core

## Offer to Help

I'm happy to:
- Share the raw `.cprof` and `callgrind.out` files if useful
- Test any optimization branches on my setup (258 BLE devices, 6 ESPHome proxies)
- Provide additional profiling data with different configurations

Thank you for this excellent integration — it's incredibly useful for room-level presence detection. These are meant as constructive observations from real-world profiling data to help make it even better.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.