Validator Performance Issue when beacon node subscribes to all subnets

### Describe the bug

The attestation miss ratio increases significantly when node subscribes to all subnets

this will likely needs multiple improvements to resolve, will open subsequent issues if we find any

### Expected behavior

Validator performance should be the same to regular node without `subscribe_to_subnet` flag

### Steps to reproduce

_No response_

### Additional context

_No response_

### Operating system

Linux

### Lodestar version or commit hash

unstable, v1.22.0