oxidecomputer / oxidecomputer/omicron

Duplicate route after switch zone receives underlay address

Open
#2,750 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

After https://github.com/oxidecomputer/omicron/pull/2713, the switch zone is assigned an underlay address:

root@oxz_switch:~# ipadm | grep omicron6
oxControlService0/omicron6 static ok    fd00:1122:3344:104::2/64

But the sled that the switch zone is located on advertises the sled prefix through ddmd:

james@frostypaws:~$ ipadm | grep sled6
underlay0/sled6   static   ok           fd00:1122:3344:104::1/64
james@frostypaws:~$ /opt/oxide/mg-ddm/ddmadm get-originated
Prefix
fd00:1122:3344:1::/64
fd00:1122:3344:104::/64    <----
fdb0:8061:5f11:ab31::/64

This causes a duplicate route to exist inside the switch zone for the sled's underlay address (in this scenario, ixgbe4 is connected to the frostypaws sled):

root@oxz_switch:~# netstat -rn -f dst:fd00:1122:3344:104::1

Routing Table: IPv6
  Destination/Mask            Gateway                   Flags Ref   Use    If   
--------------------------- --------------------------- ----- --- ------- ----- 
fd00:1122:3344:104::/64     fd00:1122:3344:104::2       U       6    1523 oxControlService0 
fd00:1122:3344:104::/64     fe80::8261:5fff:fe11:ab31   UG      1       0 ixgbe4 
default                     fd00:1122:3344:104::1       UG      2     138

This causes GENEVE packets destined for the sled to hop to oxControlService0 instead of ixgbe4. ixgbe4 is directly connected to the link used for OPTE, and this causes packets destined for an instance to not get processed by OPTE.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the switch-zone underlay address assignment and the sled-prefix advertisement through ddmd after PR 2713. Reproduce the routing table with ipadm and netstat, then verify that GENEVE traffic destined for the sled uses ixgbe4 rather than oxControlService0 and reaches OPTE.

Written by the indexing model from the issue text.

Assessment

Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.