oxidecomputer / oxidecomputer/opte

xde: mac_unicast_add failed for vnic on etherstub

Open
#373 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
77
Forks
11
Avg merge
9d 20h
Merged PRs (30d)
8

Description

My test sleds only have one physical 10G link to the fake scrimlet, but xde requires setting two devices as the underlay devices. My work around for this is to use the 10G link as the first underlay devices, and a vnic on an otherwise unconnected etherstub as the second underlay device:

pfexec dladm create-etherstub -t unattachedstub0
pfexec dladm create-vnic -t -l unattachedstub0 net1

This way, there's no ddm advertisement confusion.

This method didn't work after upgrading to OPTE 0.23.170. I saw EINVAL:

16:07:47.228Z WARN SledAgent (sprockets (BootstrapAgent)): Sled agent request failed
    err = Error managing sled agent: Could not start sled agent server: Error managing guest networking: Failure interacting with the OPTE ioctl(2) interface: command SetXdeUnderlay failed: System { errno: 14, msg: "mac_unicast_add failed for net1: 22" }

If I instead use a simnet link that's not connected to anything, or the machine's 1G link (that does also not participate in any ddm related activity), I do not see EINVAL.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the setup with an etherstub and vnic using the shown dladm commands, then trace the SetXdeUnderlay path where mac_unicast_add fails with EINVAL for net1. Done means the vnic-on-etherstub underlay works after the OPTE upgrade without that error.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.