apache / apache/cloudstack

Private vlans on a shared network not working correctly

未关闭
#13,521 12 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
component:kvm component:networking Severity:Major type:bug
主要语言
Java
星标
3.1k
派生
1.4k
平均合并
6 天 19 小时
30 天内合并 PR
32

描述

### problem

Trying to use a shared network with private vlans and the VMs were unable to ping their default gateway in testing. On investigation this turns out to be an issue with the OVS tables.

Primary vlan - 301
Isolated vlan - 302

The hypervisors use cloudbr1 which is trunk link for all guest + public traffic. All hypervisors have both vlans (301 and 302) allowed on that trunk link.

The primary vlan's L3 interface is an SVI on a Cisco switch. The Cisco switch has the correct private vlan configuration. The hypervisors can ping the L3 interface IP.

When an instance is created in this shared network it successfully gets an IP but is unable to ping the L3 interface IP. The Cisco switch shows the correct mac address of the VM NIC in it's arp table so communication to the switch is working but the return traffic is getting dropped by the OVS switch.

Looking at the OVS flow rules on the hypervisor running the VM (with some help from ai) -

root@hv2:~# ovs-ofctl dump-flows cloudbr1 | grep "dl_vlan=301"
cookie=0x0, duration=680.753s, table=0, n_packets=635, n_bytes=40640, idle_age=0, priority=200,arp,dl_vlan=301,arp_tpa=172.30.12.70 actions=strip_vlan,resubmit(,1)
cookie=0x0, duration=680.739s, table=0, n_packets=2, n_bytes=692, idle_age=659, priority=100,udp,dl_vlan=301,nw_dst=255.255.255.255,tp_dst=67 actions=strip_vlan,resubmit(,1)
cookie=0x0, duration=680.667s, table=0, n_packets=0, n_bytes=0, idle_age=680, priority=70,dl_vlan=301,dl_dst=1e:01:26:00:00:c2 actions=strip_vlan,resubmit(,1)
cookie=0x0, duration=674.094s, table=0, n_packets=105, n_bytes=6720, idle_age=21, priority=70,dl_vlan=301,dl_dst=1e:01:9d:00:00:e8 actions=strip_vlan,resubmit(,1)
cookie=0x0, duration=673.872s, table=0, n_packets=731, n_bytes=46784, idle_age=0, priority=70,dl_vlan=301,dl_dst=ff:ff:ff:ff:ff:ff actions=strip_vlan,group:301
cookie=0x0, duration=680.660s, table=1, n_packets=0, n_bytes=0, idle_age=680, priority=70,dl_vlan=301,dl_dst=1e:01:26:00:00:c2 actions=strip_vlan,output:42
cookie=0x0, duration=674.088s, table=1, n_packets=0, n_bytes=0, idle_age=674, priority=70,dl_vlan=301,dl_dst=1e:01:9d:00:00:e8 actions=strip_vlan,output:43
cookie=0x0, duration=673.865s, table=1, n_packets=0, n_bytes=0, idle_age=680, priority=70,dl_vlan=301,dl_dst=ff:ff:ff:ff:ff:ff actions=strip_vlan,group:4397

relevant lines -

cookie=0x0, duration=674.094s, table=0, n_packets=105, n_bytes=6720, idle_age=21, priority=70,dl_vlan=301,dl_dst=1e:01:9d:00:00:e8 actions=strip_vlan,resubmit(,1)
cookie=0x0, duration=674.088s, table=1, n_packets=0, n_bytes=0, idle_age=674, priority=70,dl_vlan=301,dl_dst=1e:01:9d:00:00:e8 actions=strip_vlan,output:43

table 0 has 105 matches but it is stripping the vlan 301 so when it gets to table 1 there are no matches and the default action at the end is to drop.

The solution was to add this to test -

ovs-ofctl add-flow cloudbr1 "table=1,priority=80,dl_dst=1e:01:9d:00:00:e8,actions=output:43" to match the untagged packet and send it to the VM.

Once this was added the VM could now ping it's default gateway on the Cisco switch.

But this was just a quick fix to test if that was the issue. The suggestion is that there is an issue with the matching or rewrite rules that need modifying in the code.

### versions

Cloudstack 4.22.1.0, hypervisors running Ubuntu 22.04, NFS primary and secondary storage

OVS switch version - OVS version : 2.17.9

### The steps to reproduce the bug

1. Create a shared network with a primary and isolated vlan.
2. Create an instance in the new shared network
3. Try to ping the default gateway
...

### What to do about it?

_No response_

贡献指南

打开贡献指南

调研方向

按照列出的步骤复现共享网络中的私有 VLAN 场景,然后跟踪 CloudStack 针对受影响 hypervisor 的 OVS 流生成过程。检查 table 0 和 table 1 中与 VLAN 剥离和 VM 目标 MAC 相关的规则。完成的标准是:无需手动添加未标记的 table 1 流,返回流量即可到达 VM 的网关。

由索引模型根据 Issue 内容生成。

评估

技术栈
java
领域
infrastructure, networking
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
活跃
描述清晰度
基本清楚
新手友好度
48/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。