consul agent stuck on proxycfg

#### Overview of the Issue

This is the issue from consul agent, here is the configuration(mocked)

```
{
"acl": {
"default_policy": "deny",
"down_policy": "async-cache",
"enable_token_persistence": true,
"enabled": true,
"tokens": {
"agent": "xxxxx"
}
},
"addresses": {
"http": "127.0.0.1"
},
"advertise_addr": "x.x.x.x",
"auto_encrypt": {
"tls": true
},
"bind_addr": "0.0.0.0",
"data_dir": "/var/lib/consul/",
"datacenter": "we1-prd",
"disable_keyring_file": true,
"disable_remote_exec": true,
"disable_update_check": true,
"encrypt": "EIK2bcOqT1h856tXINseeUW/xxxx",
"leave_on_terminate": false,
"limits": {
"http_max_conns_per_client": 1800,
"rpc_max_conns_per_client": 3600
},
"log_level": "TRACE",
"node_id": "x-x-x-x-x",
"node_meta": {
"kafka": "role",
"resource_group": "prd-infrastructure-central"
},
"node_name": "prd-infrastructure-central4.local",
"retry_join": [
"prd-consulserver-1.local",
"prd-consulserver-2.local",
"prd-consulserver-3.local",
"prd-consulserver-4.local",
"prd-consulserver-5.local"
],
"server": false,
"telemetry": {
"disable_hostname": true,
"prometheus_retention_time": "5m"
},
"tls": {
"defaults": {
"ca_file": "/etc/consul.d/consul-agent-ca.pem",
"verify_outgoing": true
},
"https": {
"verify_incoming": false
},
"internal_rpc": {
"verify_server_hostname": true
}
},
"ui": false
}
```

---

#### Reproduction Steps
Re-install consul, when it starts, it works normally, but after some time(may be minutes or hours), it can't response to some command like `/v1/agent/self` `consul debug`

-->

### Consul info for both Client and Server

The `consul info` command is blocked and i can't login the consul server

Here is the consul build info:

```
# consul version
Consul v1.18.2
Revision 9fc827ca
Build Date 2024-05-16T19:10:00Z
```

### Operating system and Environment details
Operating system: AlmaLinux release 8.10 (Cerulean Leopard)

Connection: Consul-agent-->consul-server

### Log Fragments
It prints on `syncing proxy services from local state` forever

```

Jan 02 05:21:12 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:12.666Z [INFO] agent.client.serf.lan: serf: EventMemberFailed: aks-x-23686585-x 10.140.11.139
Jan 02 05:21:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:24.510Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.888Z [TRACE] agent.grpc.balancer: shuffling server order: target=consul://prd.00000000-1111-2222-aee1-be366e116526/server.prd
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.888Z [DEBUG] agent.grpc.balancer: switching server: target=consul://prd.00000000-1111-2222-aee1-be366e116526/server.prd from=prd-10.60.1.2:8300 to=prd-10.60.1.1:8300
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.888Z [TRACE] agent: [core][Channel #1 SubChannel #113] Subchannel created
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.888Z [TRACE] agent: [core][Channel #1] Channel Connectivity change to CONNECTING
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.888Z [TRACE] agent: [core][Channel #1 SubChannel #111] Subchannel Connectivity change to SHUTDOWN
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.888Z [TRACE] agent: [core][Channel #1 SubChannel #113] Subchannel Connectivity change to CONNECTING
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.888Z [TRACE] agent: [core][Channel #1 SubChannel #113] Subchannel picks a new address "prd-10.60.1.1:8300" to connect
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.888Z [TRACE] agent: [core][Channel #1 SubChannel #111] Subchannel deleted
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.889Z [TRACE] agent.grpc.balancer: sub-connection state changed: target=consul://prd.00000000-1111-2222-aee1-be366e116526/server.prd server=prd-10.60.1.1:8300 state=CONNECTING
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.890Z [TRACE] agent.tlsutil: OutgoingRPCConfig: version=7
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.893Z [TRACE] agent.tlsutil: OutgoingRPCWrapper: version=7
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.893Z [TRACE] agent.tlsutil: OutgoingRPCConfig: version=7
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.894Z [TRACE] agent: [core][Channel #1 SubChannel #113] Subchannel Connectivity change to READY
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.894Z [TRACE] agent.grpc.balancer: sub-connection state changed: target=consul://prd.00000000-1111-2222-aee1-be366e116526/server.prd server=prd-10.60.1.1:8300 state=READY
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.894Z [TRACE] agent: [core][Channel #1] Channel Connectivity change to READY
Jan 02 05:21:47 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:47.897Z [DEBUG] agent.router.manager: Rebalanced servers, new active server: number_of_servers=5 active_server="prd-infrastructure-consul-server-4.local (Addr: tcp/10.60.1.3:8300) (DC: we1-prd)"
Jan 02 05:21:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:21:54.510Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:22:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:22:24.511Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:22:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:22:54.512Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:23:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:23:24.512Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:23:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:23:54.513Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:24:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:24:24.513Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:24:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:24:54.514Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:25:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:25:24.514Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:25:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:25:54.515Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:26:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:26:24.515Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:26:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:26:54.516Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:27:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:27:24.517Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:27:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:27:54.517Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:28:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:28:24.518Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:28:30 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:28:30.241Z [TRACE] agent.tlsutil: OutgoingRPCWrapper: version=7
Jan 02 05:28:30 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:28:30.242Z [TRACE] agent.tlsutil: OutgoingRPCConfig: version=7
Jan 02 05:28:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:28:54.519Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:29:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:29:24.520Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:29:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:29:54.521Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:30:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:30:24.521Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:30:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:30:54.522Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:31:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:31:24.522Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:31:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:31:54.523Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:32:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:32:24.524Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:32:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:32:54.525Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:33:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:33:24.525Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:33:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:33:54.526Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:34:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:34:24.526Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:34:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:34:54.527Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:35:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:35:24.528Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:35:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:35:54.529Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:36:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:36:24.529Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:36:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:36:54.530Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:37:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:37:24.531Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:37:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:37:54.531Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:38:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:38:24.531Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:38:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:38:54.532Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:39:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:39:24.533Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:39:54 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:39:54.534Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
Jan 02 05:40:24 prd-infrastructure-central4 consul[199933]: 2025-01-02T05:40:24.535Z [TRACE] agent.proxycfg.agent-state: syncing proxy services from local state
```