openshift / openshift/ocm-agent-operator

Address unresolved review findings in merged coderabbit-review skill

Open
#369 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ready-for-triage
Dominant language
Go
Stars
3
Forks
59
Avg merge
10h 17m
Merged PRs (30d)
25

Description

What happened

PR #363 received thorough review from both the FullSend review agent (run 34094652650) and CodeRabbit. The review agent requested changes with 1 HIGH, 2 MEDIUM, and 10 LOW findings. CodeRabbit posted 4 inline findings. PR #363 was then closed, and PR #364 — containing functionally identical code (same 3 files, same content, differing only in missing trailing newlines) — was opened by an external fork contributor. The review agent was correctly skipped on #364 (author lacks triage permissions). samanthajayasinghe approved and merged #364 without addressing any of the review findings from #363. No follow-up issue was created to track the unresolved findings.

What could go better

The merged code on master contains confirmed bugs with no tracking issue:

HIGH — config.yaml registration gap: .fullsend/harness/review.yaml is not registered in .fullsend/config.yaml under the agents: key. Confirmed: only triage and code have explicit agent entries. The review role is listed under roles: but has no source, meaning the custom review harness (with the coderabbit-review skill) may not be discovered by fullsend unless it auto-discovers harness files by role name convention. Confidence: high that the registration is missing; moderate uncertainty about whether fullsend requires explicit registration or auto-discovers by convention.

MEDIUM — documentation inaccuracies: SKILL.md Step 3 claims .coderabbit.yaml excludes hack/** and **/zz_generated.*.go, but the actual exclusions are different (build/**, .venv/**, etc.). This would cause incorrect path filtering.

MEDIUM — secrets exposure: CODERABBIT_API_KEY passed via --api-key CLI flag is visible in /proc/PID/cmdline on the CI host (runner-side only, not sandbox).

CodeRabbit findings (data-flow bugs):

  • gh api --paginate without --slurp emits one JSON array per page; multi-page responses break JSON.parse.
  • The --plain fallback passes human-readable text to a JSON-only parser; the catch block silently emits [], discarding all valid findings.
  • Issue comments lack current-head commit filtering, including stale CodeRabbit summaries.

Minor: All 3 files are missing POSIX trailing newlines.

Proposed change

Create a tracking issue on openshift/ocm-agent-operator to address the unresolved review findings from PR #363 that were merged via PR #364. The issue should enumerate the specific bugs above and prioritize: (1) verify whether .fullsend/config.yaml registration is required and add the entry if so, (2) fix the pagination bug by adding --slurp to gh api --paginate calls, (3) remove or fix the --plain fallback that silently discards findings, (4) correct the SKILL.md path exclusion documentation, (5) consider using environment variable or config file for CODERABBIT_API_KEY instead of CLI flag, (6) add trailing newlines to all 3 files.

Validation criteria

A follow-up PR addresses at least the HIGH and MEDIUM findings (config.yaml registration, documentation inaccuracies, secrets exposure) and the pagination data-flow bug. The coderabbit-review skill produces correct findings on a test PR with multi-page CodeRabbit comments.


Generated by retro agent from https://github.com/openshift/ocm-agent-operator/pull/364

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing .fullsend/config.yaml with .fullsend/harness/review.yaml and inspect SKILL.md, especially the gh api --paginate and --plain handling. Reproduce the skill against a test PR with multi-page CodeRabbit comments, then verify the required findings, documentation, secret handling, registration, and trailing newlines are corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, shell
Domain
ci-cd, devops, documentation, security, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.