apache / apache/cloudstack

bind.interface configuration only affects server binding, not client HTTP connections in cluster communication

Offen
#11,460 6 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
no-issue-activity
Vorherrschende Sprache
Java
Sterne
3.1k
Forks
1.4k
Ø Merge
6 T. 19 Std.
Gemergte PRs (30 T.)
32

Beschreibung

### problem

##### ISSUE TYPE
* Bug Report

##### COMPONENT NAME
~~~
Cluster Communication / Management Server
~~~

##### CLOUDSTACK VERSION
~~~
4.20.1.0
main branch (current)
All versions with cluster communication feature
~~~

##### CONFIGURATION
Advanced networking, VRF/multi-homed management server configuration

##### OS / ENVIRONMENT
Linux with VRF (Virtual Routing and Forwarding) or multiple network interfaces

##### SUMMARY
The `bind.interface` configuration in server.properties only affects server-side binding (Jetty HTTP server) but is completely ignored for client-side HTTP
connections used in cluster communication. This causes cluster communication failures in VRF or multi-homed environments where management traffic should be
isolated to a specific interface.

##### STEPS TO REPRODUCE
~~~
1. Configure management server with VRF or multiple interfaces
2. Set bind.interface=192.168.100.10 in server.properties (VRF interface)
3. Set cluster.node.IP=192.168.100.10 in db.properties
4. Start CloudStack management server
5. Observe cluster communication attempts
~~~

##### EXPECTED RESULTS
~~~
- Management server binds to 192.168.100.10:8080 (server-side) ✓
- Management server binds to 192.168.100.10:9090 (cluster service) ✓
- Outbound cluster HTTP connections originate from 192.168.100.10 ✓
- Cluster communication works within the same network namespace/VRF ✓
~~~

##### ACTUAL RESULTS
~~~
- Management server binds to 192.168.100.10:8080 (server-side) ✓ (edit: actually this is not true. on clean start can't bind to vrf addr)
- Management server binds to 192.168.100.10:9090 (cluster service) ✓ (edit: actually this is not true. on clean start can't bind to vrf addr)
- Outbound cluster HTTP connections originate from default interface (10.0.1.2) ❌
- Connection timeout: "Connect to 192.168.100.10:9090 [/192.168.100.10] failed: Connection timed out"

ERROR [c.c.c.ClusterServiceServletImpl] Exception from : https://192.168.100.10:9090/clusterservice
org.apache.http.conn.ConnectTimeoutException: Connect to 192.168.100.10:9090 [/192.168.100.10] failed: Connection timed out

ss output shows asymmetric routing:
SYN-SENT [::ffff:10.0.1.2]:57104 -> [::ffff:192.168.100.10]:9090
~~~

**Root Cause**: `ClusterServiceServletImpl.getHttpClient()` creates HttpClient without source address binding configuration. The HttpClient uses system
default routing instead of respecting the configured bind.interface.

**Files Affected**:
- `framework/cluster/src/main/java/com/cloud/cluster/ClusterServiceServletImpl.java:180-183`
- `client/src/main/java/org/apache/cloudstack/ServerDaemon.java:246` (server binding works correctly)

**Suggested Fix**:
Add source interface binding to HttpClient configuration by reading bind.interface from server.properties and configuring a custom ConnectionSocketFactory
with setLocalAddress().

**Example Diff**:
```diff
--- a/framework/cluster/src/main/java/com/cloud/cluster/ClusterServiceServletImpl.java
+++ b/framework/cluster/src/main/java/com/cloud/cluster/ClusterServiceServletImpl.java
@@ -177,8 +177,26 @@ public class ClusterServiceServletImpl implements ClusterService {
.setConnectionRequestTimeout(timeout)
.setSocketTimeout(timeout).build();

+ // Read bind.interface from server.properties for source binding
+ String bindInterface = getBindInterface();
+ ConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(sslContext);
+
+ if (bindInterface != null) {
+ InetAddress localAddress = InetAddress.getByName(bindInterface);
+ socketFactory = new SSLConnectionSocketFactory(sslContext) {
+ @Override
+ public Socket connectSocket(int connectTimeout, Socket sock, HttpHost host,
+ InetSocketAddress remoteAddress, InetSocketAddress localSocketAddress,
+ HttpContext context) throws IOException {
+ sock.bind(new InetSocketAddress(localAddress, 0));
+ return super.connectSocket(connectTimeout, sock, host, remoteAddress,
+ localSocketAddress, context);
+ }
+ };
+ }
+
s_client = HttpClientBuilder.create()
.setDefaultRequestConfig(config)
- .setSSLContext(sslContext)
+ .setSSLSocketFactory(socketFactory)
.build();
}

---
P.S.: Example code provided by Claude AI assistant during issue analysis.
P.P.S: Socket binding and cluster and main http is not a vrf-aware.

### versions

cloudstack 4.20.1.0
ubuntu 25

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne in framework/cluster/src/main/java/com/cloud/cluster/ClusterServiceServletImpl.java:180-183 und untersuche getHttpClient(); vergleiche anschließend sein Verhalten mit der Server-Bindung in client/src/main/java/org/apache/cloudstack/ServerDaemon.java:246. Reproduziere die VRF- oder Multi-Interface-Konfiguration aus dem Issue und überprüfe, dass ausgehende Cluster-HTTP-Verbindungen das konfigurierte bind.interface verwenden und die Kommunikation erfolgreich ist.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
java, linux
Bereich
distributed-systems, networking
Issue-Typ
Bug
Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
55/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.