oxidecomputer / oxidecomputer/omicron
[sled-agent] Use qorb for switch zone clients
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
In #7373, we had a system in a state where 1 of the 3 internal DNS servers had no records and 2 of the 3 did. sled-agent early networking code attempts to resolve dendrite in internal DNS to find the switch zone:
This appears to have failed due to accepting the response from the 1 of 3 with no data:
20:09:36.420Z WARN SledAgent (ServiceManager): Failed to look up switch zone locations
error = Error resolving dendrite services in internal DNS: no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
file = sled-agent/src/bootstrap/early_networking.rs:233
retry_after = 17.59195738s
qorb handles this more gracefully and would allow us to succeed via the responses from the other two servers; we should replace these uses of Resolver with a qorb pool.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in sled-agent/src/bootstrap/early_networking.rs, especially the resolver use around lines 233 and 264-271, and trace how switch zone locations are looked up. Replace the Resolver uses with a qorb pool so lookup succeeds when one internal DNS server returns no records while the others do; verify the early networking lookup behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100