oxidecomputer / oxidecomputer/omicron
Duplicate route after switch zone receives underlay address
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
After https://github.com/oxidecomputer/omicron/pull/2713, the switch zone is assigned an underlay address:
root@oxz_switch:~# ipadm | grep omicron6
oxControlService0/omicron6 static ok fd00:1122:3344:104::2/64
But the sled that the switch zone is located on advertises the sled prefix through ddmd:
james@frostypaws:~$ ipadm | grep sled6
underlay0/sled6 static ok fd00:1122:3344:104::1/64
james@frostypaws:~$ /opt/oxide/mg-ddm/ddmadm get-originated
Prefix
fd00:1122:3344:1::/64
fd00:1122:3344:104::/64 <----
fdb0:8061:5f11:ab31::/64
This causes a duplicate route to exist inside the switch zone for the sled's underlay address (in this scenario, ixgbe4 is connected to the frostypaws sled):
root@oxz_switch:~# netstat -rn -f dst:fd00:1122:3344:104::1
Routing Table: IPv6
Destination/Mask Gateway Flags Ref Use If
--------------------------- --------------------------- ----- --- ------- -----
fd00:1122:3344:104::/64 fd00:1122:3344:104::2 U 6 1523 oxControlService0
fd00:1122:3344:104::/64 fe80::8261:5fff:fe11:ab31 UG 1 0 ixgbe4
default fd00:1122:3344:104::1 UG 2 138
This causes GENEVE packets destined for the sled to hop to oxControlService0 instead of ixgbe4. ixgbe4 is directly connected to the link used for OPTE, and this causes packets destined for an instance to not get processed by OPTE.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the switch-zone underlay address assignment and the sled-prefix advertisement through ddmd after PR 2713. Reproduce the routing table with ipadm and netstat, then verify that GENEVE traffic destined for the sled uses ixgbe4 rather than oxControlService0 and reaches OPTE.
Written by the indexing model from the issue text.
Assessment
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100