huggingface / huggingface/serge
Referenced .ai guide files (e.g. pipelines.md) are not reliably loaded during review
- Dominant language
- Python
- Stars
- 51
- Forks
- 10
- Avg merge
- 11h 32m
- Merged PRs (30d)
- 36
Description
## Problem
`serge` only injects a single rules file — `REVIEW_RULES_PATH` (default `.ai/review-rules.md`, `reviewbot/config.py:401`) — into the system prompt under the "REVIEW RULES" block (`reviewbot/prompts.py:127-128`, via `_load_review_rules` in `reviewbot/reviewer.py:1214`).
Repos commonly split their conventions across several files and reference them from `review-rules.md`. For example, diffusers' [`.ai/review-rules.md`](https://github.com/huggingface/diffusers/blob/main/.ai/review-rules.md?plain=1#L5C1-L12C1) tells the reviewer to also read `AGENTS.md`, `models.md`, `pipelines.md`, `modular.md`, `testing.md`, etc. Those files exist in the checkout, but nothing loads them — the model is merely *told* to `read_file` them on its own initiative, and there's no step that loads them nor any check that they were read. Loading the extra guides is therefore best-effort.
## Observed impact
On the [Stable Audio 3 PR (diffusers#14119)](https://github.com/huggingface/diffusers/pull/14119), the review did not flag that `StableAudio3InpaintPipeline` inherits directly from `StableAudio3Pipeline` rather than from `DiffusionPipeline`, which violates gotcha (4) in `.ai/pipelines.md`. The review clearly had `review-rules.md` loaded and browse tools working (it even calls `StableAudio3Pipeline` "the base" of the inpaint pipeline), yet never connected that to the pipelines.md rule — consistent with `pipelines.md` not having been read.
## Likely contributing bug
Links in `review-rules.md` are relative to `.ai/` (e.g. `[pipelines.md](pipelines.md)`), but the system prompt never tells the model where the rules file lives — the header just says "from the target repo's default branch" (`reviewbot/prompts.py:127`). A model that obediently calls `read_file("pipelines.md")` hits the `not a file: pipelines.md` error (`reviewbot/tools.py:630-631`), since `read_file` resolves paths against the repo root, and may or may not recover by guessing the `.ai/` prefix. This makes the "read the sub-guides" instruction fragile in exactly the way observed.
## Proposed options (rough order of preference)
1. Resolve markdown links in `review-rules.md` and inline the referenced `.ai` files into the prompt so the guides are always loaded rather than fetched on model initiative. (Downside: grows context even when not needed — may want to do this only conditionally.)
2. Cheaper fix: add one line to the prompt stating the rules file's path, e.g. "these rules live at `.ai/review-rules.md`; relative links resolve against `.ai/`", so the read-it-yourself flow at least gets the paths right.
To confirm what happened on a given run, the run transcript emits every tool call (`emit("tool", ...)` at `reviewbot/reviewer.py:933`), so the Serge web UI / pod logs will show whether `read_file` was called on `.ai/pipelines.md`.
---
Requested by Sayak Paul - [Slack thread](https://huggingface.slack.com/archives/C065E480NN9/p1784249735500899?thread_ts=1784249735.500899&cid=C065E480NN9) - [Agent trace](https://huggingface.co/buckets/huggingface/moon-bot-memory/tree/sessions/2026-07-17T12-30-22-908Z_6eef3c19-505a-4763-a10f-10292120272a.jsonl)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.