Annotations conflict between merged gateways when configuring Ip and Subnet in azure

I have come across a peculiar behaviour when using merged gateways:

Our project aims to deploy and expose containers and databases requested by projects within our organization.
We use multiple clusters provided by AKS with each of them utilizing envoy gateway for multiple features.
Everything happens within an intranet.
On one type of cluster there are two gateways, defined as follows:

The first is general purpose, the second is for database admin UIs and has a securitypolicy attached.

The envoyproxy config:


The gatewayclass:


As you can see we define the requested IP and subnet via the infrastructure configuration on the first gateway.
These annotations are passed to the LoadBalancer service that requests the ip assignation.

Now for the aforementioned behaviour:
The load balancer service keeps alternating between the annotations from the first gateway and no annotations. The service keeps its IP because a rule forbids unexpected public ip requests but it does request one when it has no annotations.


Now I assume that it is the other gateway which has no infrastructure config that squashes the annotations.
It may be a fringe case but I have seen no indication that this kind of conf can cause conflict between merged gateways.

Chart and gateway image : 1.4
Envoy: 1.32

Solution:
A really simple fix has been to move IP configuration from the gateway to the envoyproxy: