oxidecomputer / oxidecomputer/dendrite
`tofino_stub` returns port not found when trying to apply settings to `qsfp31`
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 20
- Forks
- 3
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 4
Description
On main, attempting to apply settings to port qsfp31 fails. Little dummy test with curl that hits the dpd stub binary built for Linux:
# request body
% cat req.json
{"links":{"0":{"addrs":[],"params":{"autoneg":false,"kr":false,"lane":0,"speed":"Speed0G"}}}}
# posting to qsfp0, ..., qsfp30 is fine
% curl -X POST -H 'content-type: application/json' -H 'api-version: 12.0.0' --data @req.json 'http://[::1]:12224/port/qsfp0/settings'
{"links":{"0":{"params":{"lane":0,"speed":"Speed0G","fec":null,"autoneg":false,"kr":false,"tx_eq":null},"addrs":[]}}}
% curl -X POST -H 'content-type: application/json' -H 'api-version: 12.0.0' --data @req.json 'http://[::1]:12224/port/qsfp30/settings'
{"links":{"0":{"params":{"lane":0,"speed":"Speed0G","fec":null,"autoneg":false,"kr":false,"tx_eq":null},"addrs":[]}}}
# posting to qsfp31 fails
% curl -X POST -H 'content-type: application/json' -H 'api-version: 12.0.0' --data @req.json 'http://[::1]:12224/port/qsfp31/settings'
{
"request_id": "038b79cd-d798-41d8-8a73-90dbef511cfc",
"error_code": "invalid data: no such port",
"message": "no such port"
}
qsfp31 definitely works in the product (@sion42x has tested it on a racklette), so this appears to be a stub-only bug. I tried tracing through what's going on, and I think there's an off-by-one disagreement in Connector numbering between PortMap (which maps an API-level PortId to a Connector) and PortData (which maps a (Connector, channel) tuple to an ASIC ID):
PortMapmapsPortId::Rear(0..32)to connectorsQSFP(1..33)(but the order is scrambled) and then mapsPortId::Qsfp(0..32)to connectorsQsfp(33..65)(not scrambled:PortId::Qsfp(0)becomes connectorQSFP(33), up throughPortId::Qsfp(31)mapping to connectorQSFP(64)).PortData's map of connectors to ASIC IDs contains keys for connectorsQSFP(0..64); this does not have an entry forQSFP(64)itself.
When we try to map PortId::Qsfp(31) to an ASIC ID, PortMap tells us it's connector QSFP(64), and then we get the not found error when trying to look up QSFP(64) in PortData.
I'm very unfamiliar with all of this, but AFAICT, PortMap is shared between the stub and real ASIC, so the problem is presumably in the stub's PortData. If I change this line in asic::tofino_stub::ports::init():
+++ b/asic/src/tofino_stub/ports.rs
@@ -249,12 +249,13 @@ pub fn init() -> AsicResult<PortData> {
let eth_port = Some(CPU_PORT as u16);
connectors.insert(Connector::CPU, PhysPort::new(Connector::CPU)?);
- for id in 0..QSFP_PORT_COUNT {
+ for id in 1..=QSFP_PORT_COUNT {
let connector = Connector::QSFP(id);
let mut phys_port = PhysPort::new(connector)?;
phys_port.media = PortMedia::Optical;
I'm able to apply settings to qsfp31. But I don't know if shifting the connector indexing like this may cause other problems somewhere else in the stub.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in asic/src/tofino_stub/ports.rs, especially tofino_stub::ports::init(), and trace how PortMap and PortData represent QSFP connectors. Reproduce the curl request against the Linux dpd stub, then verify that qsfp31 succeeds without regressing the qsfp0–qsfp30 settings requests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100