NVIDIA / NVIDIA/NemoClaw

ci: classify PR Review Advisor relay outages before specialist fanout

Open
#11,603 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 43m
Merged PRs (30d)
718

Description

## Description

The PR Review Advisor can fail every specialist before review when the OpenShell execution relay is unavailable. The workflow fails closed, but no specialist review artifact is produced, so maintainers have no review evidence to inspect or narrow job to retry.

This reproduced across unrelated pull requests during the same service incident.

## Evidence

- PR #11196, exact head `81e2931906d70f6c072c00bb64731dd3ac4a8cb3`
- Run https://github.com/NVIDIA/NemoClaw/actions/runs/34657398227
- Attempts 1 and 2 both failed all nine specialists.
- Each specialist reported `transport error`, `The service is currently unavailable`, and `exec relay closed before the command reported an exit status`.
- Both attempts retained only the trusted runtime and GitHub context artifacts. No specialist review artifact was uploaded.
- PR #11588
- Run https://github.com/NVIDIA/NemoClaw/actions/runs/34658033562
- All nine specialists failed with the same transport signature and no review artifacts.

## Expected behavior

A platform-wide relay outage should produce a bounded, explicit infrastructure result that a maintainer can retry after recovery. Model or relay availability must remain advisory and must not be confused with candidate feedback or a product failure.

## Actual behavior

Every specialist fails independently, the full workflow is red, and the only retained artifacts predate specialist execution. Repeating the workflow while the incident persists consumes another full fanout without adding evidence.

## Acceptance criteria

- [ ] Detect this uniform relay-unavailable signature before or during specialist fanout.
- [ ] Preserve fail-closed review semantics and exact PR head identity.
- [ ] Publish a bounded diagnostic receipt that clearly classifies the result as infrastructure unavailability.
- [ ] Document a safe retry policy that avoids repeated full fanout while the outage persists.
- [ ] Keep Advisor and model availability advisory rather than a required merge authority.

## Checklist

- [x] I confirmed the failure on two unrelated pull requests.
- [x] I searched open issues for this exact all-specialist relay outage.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the PR Review Advisor workflow and the specialist logs from runs 34657398227 and 34658033562, using PR heads 81e2931906d70f6c072c00bb64731dd3ac4a8cb3 and PR #11588 as evidence. The work is done when uniform relay-unavailable failures produce a bounded infrastructure receipt, preserve fail-closed semantics and exact head identity, and document a safe retry policy without making Advisor or model availability merge authority.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, typescript
Domain
ci-cd
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.