provider_hidden allows vacuous passes on hardware-telemetry checks — decide legitimacy per plane

Problem

Several validations accept provider_hidden=true evidence and pass without verifying anything. The AWS reference passes ~15 observability checks and the security BMC checks this way. As written, a provider can be certified on NVSwitch/NVLink/BMC telemetry by asserting the plane is provider-owned — the check verifies nothing.

Found during the per-check requires: inventory (2026-07-21) tracing every module-suite check to its stub scripts and validation contract.

Affected checks

Observability — always/AWS provider-hidden:

• OBS07-01 EastWestNetworkTelemetryCheck, OBS08-01 ManagementNetworkTelemetryCheck, OBS09-01 NvswitchFabricTelemetryCheck
• OBS17-01 BmcSelLogsCheck, TELEM04-01 BmcGpuTelemetryCheck, TELEM05-01 StorageCapacityTelemetryCheck
• TELEM07-01 GpuNvlinkTelemetryCheck, TELEM08-01 SwitchNvlinkTelemetryCheck (hidden even in NICo)
• OBS14-01 / OBS15-01 / OBS16-01 switch logs (hidden in both AWS and NICo)
• OBS11/12/13 fabric-manager/SM/UFM logs pass hidden on AWS (real NICo UFM implementations exist)

Security:

• CNP10-01 BmcProtocolSecurityCheck emits a fixed "no customer BMC surface" pass
• SEC12-01/02/03 BMC checks pass provider-hidden on non-BM providers

Decision needed

Is "plane is provider-owned/hidden" a legitimate certification pass? Options:

1. Legitimate, but visible — keep the escape, record provider_hidden as a distinct result status ("attested hidden" vs "verified") in upload/reporting so reviewers can distinguish.
2. Hole — require real evidence; these checks become bare_metal/GPU-fabric-shaped (requires) and the escape is removed.
3. Per-plane hybrid — allowed for planes genuinely provider-owned in a cloud model (switch logs), disallowed where the declared capability implies tenant visibility (e.g. BMC/SEL and NVLink telemetry must be verified when bare_metal is declared).

Interplay with capability declarations: whatever the answer, it should be consistent with the declared-capability obligation model — a bare_metal-declaring provider passing BMC checks via "hidden" is the case that most looks like a hole.

🤖 Generated with Claude Code