Query Node Processor Slow Down

I have observed that the processor is experiencing a slow down in the rate of processing blocks/events during a re-sync.
When the processor is started up the rate is quite high (just by observing the logs). After a short while it becomes very very slow. This issue isn't really noticeable while the node is also re-indexing because the processor can only go as fast as new blocks are indexed. It is most noticeable when we only drop the processor db and keep the indexer-db intact.

After processor slows down, simply killing the processor and restarting it, gets the processor moving quickly again.
One observation is that the memory usage does grow significantly:

Is the processor perhaps not closing connections. Is the database or the indexer gateway (graphql) a bottleneck. Are those services throttling queries somehow?

I did try stopping the indexer to see if perhaps database write lock might block reading indexed blocks significantly, but that didn't seem to have any noticeable impact.

cc: @ondratra @bwhm @dmtrjsg @bedeho