Lightclient gossiping not available - OpenFailed ProtocolNotAvailable

Discussed in https://github.com/paritytech/subxt/discussions/1325

^{Originally posted by snowmead December 19, 2023}
Based on this subxt LightClient example, the gossip protocol does not seem to be available.

2023-12-19T15:24:08.272992Z DEBUG network: Gossip(local_testnet, 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp) => OpenFailed(error=Substream(Substream(ProtocolNotAvailable))) 

My configuration is based on the raw chain spec built from the template node I am running and overriding the bootnodes to connect to the locally running node.

The await on the build() function hangs and loops with the same logs shown below.

let chain_spec = match chain {
	SupportedChains::Local | SupportedChains::Compose => {
		include_bytes!("metadata/local/local.scale");
		include_str!("metadata/local/customSpecRaw.json")
	},
};

let api = LightClient::<PolkadotConfig>::builder()
	.bootnodes(chain.bootnodes())
	.max_pending_requests(NonZeroU32::MAX)
	.build(chain_spec)
	.await
	.expect("Failed to create light client");

Lightclient logs:

2023-12-19T15:24:08.158024Z  INFO libp2p_swarm: local_peer_id=12D3KooWM97vjfGBVS139zPB737Lf9gBQWsV1jg3v7dr5QQYDwFb
2023-12-19T15:24:08.158147Z DEBUG libp2p_tcp: listening on address address=0.0.0.0:33411
2023-12-19T15:24:08.158230Z DEBUG netlink_proto::handle: handle: forwarding new request to connection    
2023-12-19T15:24:08.262706Z  INFO smoldot: Chain initialization complete for local_testnet. Name: "Local Testnet". Genesis hash: 0x957e…5c38. Chain specification starting at: 0x957e…5c38 (#0)    
2023-12-19T15:24:08.262921Z DEBUG network: Slots(local_testnet) ∋ 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp    
2023-12-19T15:24:08.263020Z DEBUG json-rpc-local_testnet: JSON-RPC => {"jsonrpc":"2.0","id":"1", "method":"chain_getFinalizedHead","params":[]}    
2023-12-19T15:24:08.263130Z DEBUG runtime-local_testnet: Worker => NewDownload(block=0x957e…5c38)    
2023-12-19T15:24:08.263224Z  WARN json-rpc-local_testnet: The JSON-RPC client has just called a JSON-RPC function from the legacy JSON-RPC API (chain_getFinalizedHead). Legacy JSON-RPC functions have loose semantics and cannot be properly implemented on a light client. You are encouraged to use the new JSON-RPC API <https://github.com/paritytech/json-rpc-interface-spec/> instead. The legacy JSON-RPC API functions will be deprecated and removed in the distant future.    
2023-12-19T15:24:08.263257Z DEBUG runtime-local_testnet: Worker <= Reset(finalized_block: 0x957e…5c38)    
2023-12-19T15:24:08.263346Z DEBUG network: Connections(12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp) <= StartConnecting(remote_addr=/ip4/127.0.0.1/tcp/30333, local_peer_id=12D3KooWNTPGrD9hc9CBoYbbueaCyUTcRTP4T2ZV3KHAJmTXD7u1)    
2023-12-19T15:24:08.271649Z DEBUG network: Connections(12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp, /ip4/127.0.0.1/tcp/30333) => HandshakeFinished    
2023-12-19T15:24:08.271748Z DEBUG network: Gossip(local_testnet, 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp) <= Open    
2023-12-19T15:24:08.272992Z DEBUG network: Gossip(local_testnet, 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp) => OpenFailed(error=Substream(Substream(ProtocolNotAvailable)))    
2023-12-19T15:24:08.273043Z DEBUG network: Slots(local_testnet) ∌ 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp (ban_duration=15s)    
2023-12-19T15:24:08.773370Z DEBUG network: Connections(12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp) => IdentifyRequest    
2023-12-19T15:24:13.263114Z DEBUG network: Discovery(local_testnet) => NoPeer    
2023-12-19T15:24:18.263177Z  WARN sync-service-local_testnet: GrandPa warp sync idle at block #0 (0x0x957e…5c38)    
2023-12-19T15:24:23.263347Z DEBUG network: Discovery(local_testnet) => NoPeer    
2023-12-19T15:24:23.273360Z DEBUG network: Slots(local_testnet) ∋ 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp    
2023-12-19T15:24:23.273483Z DEBUG network: Gossip(local_testnet, 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp) <= Open    
2023-12-19T15:24:23.274518Z DEBUG network: Gossip(local_testnet, 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp) => OpenFailed(error=Substream(Substream(ProtocolNotAvailable)))    
2023-12-19T15:24:23.274555Z DEBUG network: Slots(local_testnet) ∌ 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp (ban_duration=15s)    
2023-12-19T15:24:28.263383Z  WARN sync-service-local_testnet: GrandPa warp sync idle at block #0 (0x0x957e…5c38)

And I have this warning message regarding the RPC api used to query chain_getFinalizedHead.

The legacy RPC call is being utilized instead of the new one. Is there some flag or configuration I need to specify to fix this, or is it simply not implemented in the library yet?

2023-12-19T15:24:08.263224Z  WARN json-rpc-local_testnet: The JSON-RPC client has just called a JSON-RPC function from the legacy JSON-RPC API (chain_getFinalizedHead). Legacy JSON-RPC functions have loose semantics and cannot be properly implemented on a light client. You are encouraged to use the new JSON-RPC API <https://github.com/paritytech/json-rpc-interface-spec/> instead. The legacy JSON-RPC API functions will be deprecated and removed in the distant future.    
2023-12-19T15:24:08.263257Z DEBUG runtime-local_testnet: Worker <= Reset(finalized_block: 0x957e…5c38)

Start with the LightClient example and the builder configuration shown in the issue, then trace the gossip connection and chain_getFinalizedHead calls that produce the logged failures. Reproduce against the local template node; done means the light client can complete build and connect without repeated ProtocolNotAvailable failures, with the RPC warning behavior addressed or explained.