Huge network thread latency to/from main thread when subscribing to all subnets

### Describe the bug

When subscribing to all subnets, network thread to/from main thread latency increase significantly

I don't see any latency from libp2p, or at least it's a tiny difference, we still receive gossip block on time at network thread

the above latency leads to significantly increase of below topics:
- Avg block recv to gossip validation delay

- Avg block recv to state transition delay

### Expected behavior

for beacon_block topic, we don't want to have this latency, as it causes block to becomes head very late and validator votes for wrong head, see #7186

### Steps to reproduce

_No response_

### Additional context

_No response_

### Operating system

Linux

### Lodestar version or commit hash

v1.22.0, unstable