[netty] Refuse connections from certain clients (based on IP)

Hello there. I have a need to block certain clients from establishing a connection to the server (i.e. at the moment of tls handshake, based on their ip address)
I know a `ServerInterceptor` would work here, but I don't really want to make redundant checks every time a client executes a rpc

So far I've managed to proceed with proxy `X509ExtendedTrustManager` - I am able to close any inbound/outbound traffic during tls handshake.
But I am unable to retrieve ip address of the client from `SSLEngine` (`peerhost`/`peerport` is set to null there).
A little bit of debugging and I see that `io.grpc.netty.shaded.io.grpc.netty.ProtocolNegotiators.ServerTlsHandler.handlerAdded()` instantiates a new `SSLEngine` without setting `peerhost`/`peerport`, though it has access to `channel` (which contains `remoteAddress`) from `ChannelHandlerContext` (curious whether it was done intentionally, if not and this is the only option I have left then I'm more than willing to make contribution to address it)

So, at this point I'm out of ideas what to do. Could anyone please guide me what my options are?
Thanks