apache / apache/cloudstack

VPC private gateway cannot be created on a VXLAN-isolated physical network

Open
#14,143 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.1k
Forks
1.4k
Avg merge
6d 19h
Merged PRs (30d)
32

Description

### problem

Creating a VPC private gateway with a vxlan:// broadcast URI fails:

unsupported type of broadcastUri specified: vxlan://1005002

createPrivateGateway documents its `vlan` parameter as "the network implementation uri for the
private gateway", and a VXLAN-isolated physical network hands out VNIs rather than 802.1Q tags,
so vxlan:// is the correct value to pass.

It is rejected by an allowlist in NetworkServiceImpl.createPrivateNetwork() that admits only two
schemes:

URI uri = BroadcastDomainType.fromString(broadcastUriString);
uriString = uri.toString();
BroadcastDomainType tiep = BroadcastDomainType.getSchemeValue(uri);
// numeric vlan or vlan URI are ok for now
// TODO make a test for any supported scheme
if (!(tiep == BroadcastDomainType.Vlan || tiep == BroadcastDomainType.Lswitch)) {
throw new InvalidParameterValueException("unsupported type of broadcastUri specified: " + broadcastUriString);
}

The "TODO make a test for any supported scheme" comment directly above suggests the list was
always meant to be provisional.

The rest of the path already copes with VXLAN, so this is the only functional blocker:

* encodeVlanIdIntoBroadcastUri() preserves an explicit vxlan:// prefix unchanged.
* The VNI overlap checks use BroadcastDomainType.getValue(), which is scheme-agnostic.
* NicProfileHelperImpl.createPrivateNicProfileForGateway() already passes the whole URI and
derives the broadcast type from its scheme.
* BridgeVifDriver derives the protocol from the URI scheme rather than the broadcast type, so it
already selects modifyvxlan.sh and the brvx- bridge.
* No systemvm/VR script references vlan at all - the private gateway interface is keyed on
MAC/device and the is_private_gateway flag, and the broadcast URI is never sent to the VR.
* No schema change is needed: networks.broadcast_uri, nics.broadcast_uri and
vpc_gateways.vlan_tag (which despite its name already stores the full URI) are all
varchar(255), and vxlan://16777214 is 18 characters.

There is one related data-consistency problem that should be fixed in the same change.
NetworkOrchestrator.createGuestNetwork() encodes the URI correctly and then unconditionally sets
the type:

userNetwork.setBroadcastUri(uri);
if (!vlanIdFinal.equalsIgnoreCase(Vlan.UNTAGGED)) {
userNetwork.setBroadcastDomainType(BroadcastDomainType.Vlan);

Guest networks are corrected afterwards by VxlanGuestNetworkGuru.design(), but that guru excludes
system-only offerings (&& !offering.isSystemOnly()) so it never sees a private gateway, and
PrivateNetworkGuru has no equivalent override. The network would therefore be persisted with
broadcast_uri = 'vxlan://' and broadcast_domain_type = 'Vlan'. The gateway still works,
because nothing on the KVM bridge path reads that column, but the row is inconsistent and would
mislead any future code that trusts it.

### versions

CloudStack: originally hit on 4.20.1.0.

The guard is byte-identical on 4.20.1.0, 4.20.2.0, 4.20.3.1, 4.21.0.0, 4.22.0.0, 4.22.1.1 and
current main (verified at b0601e5478 and 4bfeb96c96), so all supported releases are affected.
Not a regression.

Hypervisor: KVM with the Linux bridge VIF driver (network.bridge.type=native).
Network: Advanced zone, physical network with VXLAN isolation method.

### The steps to reproduce the bug

1. In an Advanced zone, configure a physical network with isolation method VXLAN and a guest VNI
range (e.g. 1000000-1010000).
2. Create a VPC.
3. As root admin, add a private gateway with a VXLAN broadcast URI, choosing a VNI outside the
guest VNI range:

create privategateway vpcid= physicalnetworkid= \
vlan=vxlan://1005002 \
ipaddress=10.10.10.2 gateway=10.10.10.1 netmask=255.255.255.0

Expected: the private gateway is created on VNI 1005002 and the VR NIC is attached to the
corresponding VXLAN bridge (brvx-1005002).

Actual: the API fails immediately with

unsupported type of broadcastUri specified: vxlan://1005002

Note that passing a bare number (vlan=1005002) is not a workaround: NetworkServiceImpl wraps any
value that does not already contain "://" as vlan://, so it is silently accepted as an
802.1Q tag of 1005002 rather than a VNI.

### What to do about it?

Two changes, both in the server module.

1. NetworkServiceImpl.createPrivateNetwork() - accept BroadcastDomainType.Vxlan alongside Vlan
and Lswitch. This is the functional fix.

- if (!(tiep == BroadcastDomainType.Vlan || tiep == BroadcastDomainType.Lswitch)) {
+ if (!(tiep == BroadcastDomainType.Vlan || tiep == BroadcastDomainType.Vxlan || tiep == BroadcastDomainType.Lswitch)) {

2. PrivateNetworkGuru.design() - derive the broadcast domain type from the URI scheme so the
persisted row is self-consistent, the same correction VxlanGuestNetworkGuru.design() already
makes for guest networks. No-op for vlan://.

if (userSpecified.getBroadcastUri() != null) {
network.setBroadcastUri(userSpecified.getBroadcastUri());
+ network.setBroadcastDomainType(BroadcastDomainType.getSchemeValue(userSpecified.getBroadcastUri()));
network.setState(State.Setup);
}

Plus unit test coverage in CreatePrivateNetworkTest for a vxlan:// URI.

Contributor guide

Open the contributing guide

Research direction

Start with NetworkServiceImpl.createPrivateNetwork() and the allowlist for broadcast URI schemes, then inspect PrivateNetworkGuru.design() and the existing VxlanGuestNetworkGuru.design() correction. Run or extend CreatePrivateNetworkTest for a vxlan:// URI. Done means private gateways accept VXLAN URIs, the persisted broadcast domain type matches the URI scheme, and existing vlan:// behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, cloud, networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.