Add XTS BN coverage for delayed BNs

Test Setup

1. Deploy n BNs, where n is equal to 1/3 the number of CNs. Configure BNs to be able to backfill from each other
2. Configure 1/3 of CNs to pointing to a primary BN, with other BNs listed as backup options.
3. Kick off SDLT/MDLT at 200 TPS, let run for about 3 hours. TPS here should be configureable variable.
4. Scale up to 2k TPS. TPS here should be configureable variable.
5. After 6 hours of total run, scale down BNs for about 5 mins, then scale them back up. This will ensure the BNs fall hind some period then start up will force fall behind. Number of BNs to scale down should be configureable.
   • kubectl scale statefulsets <$NAMESPACE>-block-node-server -n $NAMESPACE --replicas=0
   • kubectl scale statefulsets <$NAMESPACE>-block-node-server -n $NAMESPACE --replicas=1
6. After 7 hours of total run, dynamically change block-nodes.json on the CNs to be empty (essentially disable block streaming). TPS here should be configureable variable.
7. Collect logs

Configureable items

1. Initial TPS
2. Ramped up TPS
3. Number of BNs to scale down and up
4. Whether to clear block-node.json file

On first iteration we can hardcode configs but eventually we should allow variability as it will ensure we can continue to use the test for growing scenarios.

• Note this may require a new Solo version for multi CN <> BN support