[Network] long delay between receiving a block validation request and getting a response

The foundation nakamoto testnet miner sees this:

INFO [1729694751.965402] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:346] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: beginning to watch for block signatures OR posted blocks., threshold: 1020
INFO [1729694756.336104] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 03157271cca37f5ebb4147aad30538fb4b33392ec1cc7161de5f815e29d5146041, signer_slot_id: 2, signature: 01bcc792e49d83b4d7896180dc79bbf19f97eace67d43beda226b3c9a3bb21ae62548e88fb9ec46d10953fafd4ecb173bfd70881229474a31f2558ef1e2953ce95, signer_weight: 1, total_weight_signed: 1, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe64603f54dae48558a5d82d9bd885e97a01, release build, linux [x86_64])
INFO [1729694756.692839] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 03fbbbc6f26cb4105faf48008f8d7ef1900943a96fdbf63a95b94d7f2d87a4e9bb, signer_slot_id: 4, signature: 01b350643ec788493b0bf1770bddf75b4fc02ac1c7ef09537b2bd42c7029928d4b3311421103ca6c2c55442dd4c1052013ee4c3ae06e6baceb08625aa0665ff393, signer_weight: 1, total_weight_signed: 2, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])
INFO [1729694785.760468] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 026ddf4e4e144eac1aad4a215641feee86386a1bff3776f981430364d0ae89cd36, signer_slot_id: 0, signature: 00bd530db16aef6ee107a3d04a471b787618d958191fcc8d8e13ff59f7c769a3aa74b5865fe630ec1dad8bbcc446c0a211606f14d95e3e3fbb82f18ca3b852cb64, signer_weight: 485, total_weight_signed: 487, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])
INFO [1729694788.199587] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 02e77164ae21938de2058eb349353f0ae2ac3b76e1a69a03ea94c6fc352164c467, signer_slot_id: 1, signature: 008b9f244f4f92588bdde7a252779b6bf08c4f0fdfa54c19743b2805ee494d40e10bb3d7c93e17ac9a52c23bd37979f55cf2274b289be47912f9e9b73d2fc8c29e, signer_weight: 485, total_weight_signed: 972, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])
INFO [1729694788.199752] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:511] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Signature Added to block, block_signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signer_pubkey: 0343b909be3379621d5e68a883e6c74c2222666f4bf6cc38fade2c9d1c57d92213, signer_slot_id: 3, signature: 010a760eb984e3388b3ba03b70a7923cba7e3ded06b4a1de71d7a1bfed723c63c51e3de0f6359b1f9803a91773221d764df160e892d81b5ecfd656f3964384516a, signer_weight: 485, total_weight_signed: 1457, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, server_version: stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])
INFO [1729694788.199787] [testnet/stacks-node/src/nakamoto_node/sign_coordinator.rs:594] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] SignCoordinator: Received enough signatures. Continuing., stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e
INFO [1729694788.569665] [testnet/stacks-node/src/nakamoto_node/miner.rs:404] [miner.ef7cfd2050ac8b78acb983cd879d0dac12faaf30ab4e3ef400180de87b7809a7] Miner: Block signed by signer set and broadcasted, signer_sighash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, stacks_block_id: 77b4a778a0072faae62164caf40576cea7fb3828c9d897d423bbe8389703133e, block_height: 38268, consensus_hash: d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0

The hiro nodes and signers see this:

10/23/2024, 02:53:09.682 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Downloaded
10/23/2024, 02:46:56.231 PM	testnet-stacks-blockchain-nakamoto-3-0 - Relayer: Submitted block-commit
10/23/2024, 02:46:35.321 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:34.463 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:34.428 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:33.235 PM	testnet-seed-nakamoto-0 - Advanced to new tip! d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:33.128 PM	testnet-pool-nakamoto-0-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.899 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: Got block pushed message
10/23/2024, 02:46:31.846 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: Got block pushed message
10/23/2024, 02:46:31.814 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.784 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: Got block pushed message
10/23/2024, 02:46:31.631 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.569 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.511 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:31.457 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:30.924 PM	testnet-pool-nakamoto-0-0 - Advanced to new tip! d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:30.825 PM	testnet-pool-nakamoto-0-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:30.473 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.966 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.906 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.821 PM	testnet-seed-nakamoto-0 - Stored incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:29.602 PM	testnet-seed-nakamoto-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.266 PM	testnet-pool-nakamoto-0-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:29.257 PM	testnet-stacks-blockchain-nakamoto-3-0 - Advanced to new tip! d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:29.209 PM	testnet-pool-nakamoto-0-0 - Stored incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:29.036 PM	testnet-pool-nakamoto-0-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Pushed
10/23/2024, 02:46:27.150 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:27.116 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:27.079 PM	testnet-stacks-blockchain-nakamoto-3-0 - Stored incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178
10/23/2024, 02:46:26.926 PM	testnet-stacks-blockchain-nakamoto-3-0 - Handle incoming Nakamoto block d3f81e962f1c9bbc1fe2fe3d434d2b19af0010e0/62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178 obtained via Uploaded
10/23/2024, 02:46:23.627 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: Broadcasting a block response to stacks node: Accepted(BlockAccepted { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signature: 010a760eb984e3388b3ba03b70a7923cba7e3ded06b4a1de71d7a1bfed723c63c51e3de0f6359b1f9803a91773221d764df160e892d81b5ecfd656f3964384516a, metadata: SignerMessageMetadata { server_version: "stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])" } })
10/23/2024, 02:46:23.587 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: Broadcasting a block response to stacks node: Accepted(BlockAccepted { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signature: 008b9f244f4f92588bdde7a252779b6bf08c4f0fdfa54c19743b2805ee494d40e10bb3d7c93e17ac9a52c23bd37979f55cf2274b289be47912f9e9b73d2fc8c29e, metadata: SignerMessageMetadata { server_version: "stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])" } })
10/23/2024, 02:46:23.579 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: Broadcasting a block response to stacks node: Accepted(BlockAccepted { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, signature: 00bd530db16aef6ee107a3d04a471b787618d958191fcc8d8e13ff59f7c769a3aa74b5865fe630ec1dad8bbcc446c0a211606f14d95e3e3fbb82f18ca3b852cb64, metadata: SignerMessageMetadata { server_version: "stacks-signer 3.0.0.0.0 (release/3.0.0.0.0:dd1ebe6, release build, linux [x86_64])" } })
10/23/2024, 02:46:23.529 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: Received a block validate response: Ok(BlockValidateOk { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, cost: ExecutionCost { write_length: 2676198, write_count: 1227, read_length: 1099372, read_count: 1207, runtime: 76922159 }, size: 498559 })
10/23/2024, 02:46:23.516 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: Received a block validate response: Ok(BlockValidateOk { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, cost: ExecutionCost { write_length: 2676198, write_count: 1227, read_length: 1099372, read_count: 1207, runtime: 76922159 }, size: 498559 })
10/23/2024, 02:46:23.502 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: Received a block validate response: Ok(BlockValidateOk { signer_signature_hash: 62807f1655cd545c79edd1fb65b07b147c083dbe9ed1072b6cd56d51078b4178, cost: ExecutionCost { write_length: 2676198, write_count: 1227, read_length: 1099372, read_count: 1207, runtime: 76922159 }, size: 498559 })
10/23/2024, 02:46:23.494 PM	testnet-stacks-blockchain-nakamoto-3-0 - Participant: validated anchored block
10/23/2024, 02:46:22.486 PM	testnet-stacks-blockchain-nakamoto-3-0 - Received block proposal request
10/23/2024, 02:46:21.392 PM	testnet-stacks-blockchain-nakamoto-3-0 - Received block proposal request
10/23/2024, 02:46:21.370 PM	testnet-stacks-blockchain-nakamoto-3-0 - Received block proposal request
10/23/2024, 02:45:53.205 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #1: received a block proposal for a new block. Submit block for validation. 
10/23/2024, 02:45:53.141 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #0: received a block proposal for a new block. Submit block for validation. 
10/23/2024, 02:45:53.072 PM	testnet-stacks-blockchain-nakamoto-3-0 - Cycle #4 Signer #3: received a block proposal for a new block. Submit block for validation. 

Why is there such a long delay between the signer submitting a block proposal and the node receiving it?