Electrumx keeps crashing during it's initial sync with Bitcoind. Out of memory?

Electrumx keeps crashing during it's initial sync with Bitcoind, seems memory related.
I'm running ElectrumX 1.15.0 on a Raspberry Pi4 with 4GB RAM, Raspbian 4.19.97.
Other running applications: Bitcoind 0.20.0 as backend, and LND 0.10.1 taking up some memory.
This is the error:

Jul 01 20:50:47 Pi electrumx_server[24345]: INFO:BlockProcessor:our height: 339,907 daemon: 637,215 UTXOs 1,621MB hist 582MB                                                                                       
Jul 01 20:51:47 Pi electrumx_server[24345]: INFO:BlockProcessor:our height: 340,004 daemon: 637,215 UTXOs 1,624MB hist 601MB                                                                                       
Jul 01 20:51:49 Pi electrumx_server[24345]: INFO:DB:flushed filesystem data in 1.89s                                                                                                                               
Jul 01 20:52:04 Pi electrumx_server[24345]: terminate called after throwing an instance of 'std::bad_alloc'                                                                                                        
Jul 01 20:52:04 Pi electrumx_server[24345]:   what():  std::bad_alloc                                                                                                                                              
Jul 01 20:52:05 Pi systemd[1]: electrumx.service: Main process exited, code=killed, status=6/ABRT                                                                                                                  
Jul 01 20:52:05 Pi systemd[1]: electrumx.service: Failed with result 'signal'.                                                                                                                                     
Jul 01 20:52:10 Pi systemd[1]: electrumx.service: Service RestartSec=5s expired, scheduling restart.                                                                                                               
Jul 01 20:52:10 Pi systemd[1]: electrumx.service: Scheduled restart job, restart counter is at 1.                                                                                                                  
Jul 01 20:52:10 Pi systemd[1]: Stopped Electrumx server daemon.

I've set CACHE_MB = 2000, and then 3000, this delays the crash significantly but does not prevent it.
During sync I can see the process eating up my RAM over the course of an hour, when the system hits around 3.3g out of 3.81g total RAM the process crashes and the memory is freed up.

The process does not seem to release any RAM it uses even after succesfully flushing filesystem data several times before eventually crashing.

Start with the reported BlockProcessor and DB log entries, reproducing the initial sync on the Raspberry Pi 4 configuration described in the issue while monitoring memory. Trace the growing UTXOs and history usage around filesystem flushes; done means initial sync completes without the process reaching the device's memory limit or crashing with std::bad_alloc.