influxdata / influxdata/docs-v2

PR workflow: Improve review status signals and reduce noise

Open
#6,937 0 comments 1 reaction 2 assignees Claimed by @jstirnaman View on GitHub
Dominant language
JavaScript
Stars
82
Forks
326
Avg merge
1d 1h
Merged PRs (30d)
82

Description

PR visual review and related workflow issues

## Summary

During the v1.12.3 release (PR #6872), several PR review workflow issues surfaced that reduce the signal-to-noise ratio for human reviewers.

## Problems

### 1. `copilot-visual-review` silently skips and reports SUCCESS

Is it running? `@copilot` is invoked with comments when it appears first in the comment. The template appears to invoke it from further down in the body. Is it actually working?

The `copilot-visual-review` CI check can pass without actually performing a review. On PR #6872, the check reported SUCCESS but no visual review comment
was posted. This is a false positive — the green checkmark implies review happened when it didn't.

Related task: https://github.com/influxdata/docs-v2/tasks/d9d0ed4e-26ae-4bbf-8ce3-f9bc71daf79a

**Expected behavior:** If the visual review doesn't run, the check should report as skipped or failed, not SUCCESS.

### 2. Preview URLs listed redundantly across multiple comments

PR #6872 had preview URLs listed in three separate bot comments:
- PR Preview Action comment
- PR Preview detail comment (pages list)
- Doc Review visual review prompt

This makes the PR noisy and harder for humans to find the actual status of checks and reviews.

**Suggested fix:** Consolidate preview URL listings into a single comment, or have later comments reference the first one instead of duplicating.

### 3. No clear completion signal for visual review

When visual review does run (or is performed manually), there's no standardized comment or artifact that signals "visual review complete." Humans
scanning the PR can't easily tell whether it happened.

**Suggested fix:** Agents/checks that perform reviews should post a clearly identified completion comment (e.g., "Visual Review — [Agent Name]: No
issues found" or "Visual Review — [Agent Name]: 2 issues found"). The agent identity and outcome should be immediately visible.

### 4. Agent personas lack clear identification

Multiple bot comments from `github-actions` make it hard to tell which workflow produced which comment. Each automated review step should have a
distinct, recognizable name so humans can quickly scan for what ran and what didn't.

## Context

- PR #6872: https://github.com/influxdata/docs-v2/pull/6872
- CI run with link-checker failures: https://github.com/influxdata/docs-v2/actions/runs/23058520107
- Copilot opened a separate PR (#6932) instead of posting visual review comments on #6872

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.