Add answers to these questions on WebSockets to docs

How API gateway can close the connection if the client and server is idle more than an 5 hours?

The library used for WS communication is https://netty.io/wiki/user-guide-for-4.x.html. The default behavior does not implement any mechanism how to handle idle connection (which is fallen to default behavior i.e infinite), But if there is a requirement that should treat the idle connection specific way that may be an extension that needs to implement by utilzing https://netty.io/4.0/api/io/netty/handler/timeout/IdleStateHandler.html.

How the backend application will trust the API gateway WebSocket request and allow the connection?
What kind of Authentication we can use to establish connection between gateway and backend service?

Websockets (WS) BE (Backend) initiated connections can be trusted through the GW outbound synapse extensions. (Depending on the method team would prefer to validate such request such as verifying the signature of the incoming requests from BE with combination of specific headers, similar to how WebSub APIs BE channels secured with APIM 4.0.0) Please note WSO2 APIM WS released feature don't have UI to embed synapse extensions but, based on how team would like to proceed, we may need to check how to improve the WS API from publisher point of view to incorporate extensions per API.

How client can re-establish connection when gateway node is down and what kind of error messages , error code will be displayed in this scenario?

Client connection would automatically connect through a netty pipe via a duplex handler which allows that to exchange the websocket frame (as depicted in the diagram) refer [1]. Team is internally verifies whether any specific code that we need to monitor.

What kind of data can server/client send or exchange?

Netty.io should be capable of handling most of the streaming scenario, be specific what sort of use case you may look at from data exchange angle.

Start with the questions and the linked Netty user guide and IdleStateHandler reference. Verify the gateway's documented behavior for idle connections, backend authentication, node failure and reconnection, error reporting, and exchanged data. Done means adding accurate answers for each question to the relevant WebSocket documentation, with any limitations clearly stated.