ColoredCow / ColoredCow/engineering-recipes
fix: Claude intermittently skips posting review comment on PR
- Dominant language
- Shell
- Stars
- 1
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
## Problem
The Claude AI code review workflow sometimes generates the review but **never posts it as a comment**. The workflow completes successfully (green checkmark), the review exists in workflow logs, but no comment appears on the PR. The developer has no way to know the review was generated — it looks like the bot didn't run.
## Evidence
**PR:** [coloredcow-admin/coloredcow#1152](https://github.com/coloredcow-admin/coloredcow/pull/1152)
**Workflow Run:** [Run 23399458396 (08:46 UTC, 22 March 2026)](https://github.com/coloredcow-admin/coloredcow/actions/runs/23399458396)
### Context — 4 runs on the same PR
| Run | Time | Link | Comment Posted? |
|-----|------|------|-----------------|
| 1 | 08:04 | [Run 23398816364](https://github.com/coloredcow-admin/coloredcow/actions/runs/23398816364) | Yes |
| 2 | 08:29 | [Run 23399208437](https://github.com/coloredcow-admin/coloredcow/actions/runs/23399208437) | Yes |
| **3** | **08:46** | **[Run 23399458396](https://github.com/coloredcow-admin/coloredcow/actions/runs/23399458396)** | **No — review lost** |
| 4 | 09:51 | [Run 23400517185](https://github.com/coloredcow-admin/coloredcow/actions/runs/23400517185) | Yes |
Code was changed between Run 3 and Run 4 (new commit pushed), but the commit does not trigger the review — only the `ready for review` label does. The same trigger with the same workflow sometimes posts and sometimes doesn't.
### What Claude did in each run (from workflow logs)
- **Run 1:** Claude called `gh pr comment` → comment appeared
- **Run 3 (failed):** Claude only ran `gh pr view` and `gh pr diff`, generated the review, but **never called `gh pr comment`** — returned review as text only
- **Run 4:** Claude called `gh pr comment` → comment appeared
### The review that was never posted (from Run 3 logs)
```
## Code Review Summary
### Critical Issues
None
### Suggestions
- ai-crawler-logger.php:61-62 — Validate REQUEST_URI length (max 255 chars)
- ai-crawler-logger.php:72 — Consider periodic cleanup for old logs
- ai-crawler-logger.php:14 — Consider sampling or template_redirect hook
- robots.txt:1-2 — Document that physical file intentionally overrides Yoast
### What Looks Good
- Security: Proper nonce checks, capability verification, sanitization
- WordPress Standards: array() syntax, proper hooks, if():/endif;
- No autoload, clear separation, scope adherence
```
This full review was generated and exists in workflow logs — but the developer never saw it.
### Impact
- Review **invisible** to the team — developer thinks the bot didn't run
- **$0.10 wasted** per failed run (API cost with zero visible output)
- Developer has to **re-trigger and wait** again
## Root Cause
The prompt in `claude-code-review.yml` says:
> "Create ONE single comment with your complete review"
But doesn't explicitly instruct Claude to use `gh pr comment`. Since LLMs are non-deterministic (same input can produce different behavior each run), Claude sometimes calls `gh pr comment` to post the review, and sometimes just returns the review as text without posting.
The `use_sticky_comment: true` setting was supposed to be a fallback — auto-post Claude's result as a comment after execution. But it's not working for the `labeled` event trigger, so when Claude skips posting, there is no safety net.
## How the system currently works
**Expected flow:**
Developer adds label → workflow triggers → Claude reviews code → posts comment → developer sees review
**What actually happens (intermittently):**
Developer adds label → workflow triggers → Claude reviews code → **decides on its own** whether to call `gh pr comment` or just return text → sometimes posts, sometimes doesn't → if it doesn't post, review is lost in workflow logs
## File Reference
- `.github/workflows/claude-code-review.yml` — the `prompt` input default value and `use_sticky_comment` configuration
- For More details, check this [doc file](https://docs.google.com/document/d/1nMY5In7WSbIvN0HZ-nnlhHlUWcsLQRQnteQTLXRM7gk/edit?tab=t.impait1nzi1n).
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.