MetPX / MetPX/sr_insects

static_flow API tests silently do not run on pull requests from forks

Open
#93 0 comments 0 reactions 1 assignee View on GitHub

@robjarawan is already working on this.

Since Sep 8, 2026.

bug Gap
Dominant language
Shell
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

`static_flow/flow_maint_test.sh` fetches three example scripts from `MetPX/sarracenia` at `${GITHUB_HEAD_REF}`:

```bash
wget https://raw.githubusercontent.com/MetPX/sarracenia/${GITHUB_HEAD_REF}/sarracenia/examples/moth_api_consumer.py
```

`GITHUB_HEAD_REF` is set on `pull_request` events only, and it names a branch in the **head** repository. The URL hardcodes the owner as `MetPX`. So for a pull request raised from a fork, that ref resolves only if MetPX happens to carry a branch of the same name. Usually it does not.

#### What that costs

Two things, and the second is the one that concerns me more.

**The check is permanently red for fork contributors, and it tests nothing.** All three fetches 404, `moth_api_producer.py` is never written, and tests 2 and 3 fail. They do not fail because something is wrong with the pull request — they fail before any SR3 code executes. So on every fork pull request that check is noise, and the Python API path it exists to cover is not exercised at all.

From `MetPX/sarracenia` run [33986304253](https://github.com/MetPX/sarracenia/actions/runs/33986304253), 2026-09-05, on both matrix legs:

```text
--2026-09-05 22:56:55-- https://raw.githubusercontent.com/MetPX/sarracenia/fix/sender-dry-run-local-dir/sarracenia/examples/moth_api_consumer.py
HTTP request sent, awaiting response... 404 Not Found
python3: can't open file '.../moth_api_producer.py': [Errno 2] No such file or directory
test 2 FAILURE: moth_api_consumer.py example should consume 5 messages.
test 3 FAILURE: blank results! flow_api_consumer.py should consume 5 messages, found
```

A `push` of that same commit `3d95cd91` passes, because `GITHUB_HEAD_REF` is empty there and the script falls back to `development`. Same code, same commit, different event.

Since `29b03a96` there have been 30 `pull_request` runs of `flow_basic.yml`, of which 14 failed. All the failures are fork pull requests. Most are mine, but `Maksim-Burtsev` and `mmaxjr` have raised fork pull requests over the same period.

**Where a branch name does collide, it fails silently instead.** If a fork's head branch happens to share a name with a MetPX branch — and people do raise pull requests from a branch called `development` — the fetch succeeds, pulls MetPX's copy of the examples, and the test passes on content that is not the pull request's. That is a wrong answer rather than a missing one, and nothing in the log says so.

#### Where it came from

`29b03a96` moved this from `GITHUB_BASE_REF` to `GITHUB_HEAD_REF` so a pull request would exercise its own examples. That is the right intent and it works for same-repository pull requests. It just cannot work through a hardcoded owner when the head branch lives in a fork.

Line 50 also has `if [ ! "${GITHUB_HEAD_REF}"]` — missing a space before the bracket. It raises `[: missing ']'` on every pull request and makes the file unparseable by `shellcheck`. It happens not to change the outcome, but it is in the logs above.

#### What I propose

Under Actions the fetch is unnecessary: the sarracenia checkout is already on the runner at `${GITHUB_WORKSPACE}`, and `travis/flow_autoconfig.sh` pip installs from that same tree. Taking the examples from there tests the pull request's own examples for fork and same-repository pull requests alike, guarantees the examples and the installed library are the same revision, and removes the network from the common path entirely.

Keep fetching for anyone running the script outside Actions, but settle the ref once rather than per file, so a genuinely deleted or renamed example still fails loudly instead of being quietly replaced by the `development` copy.

Happy to be told the simpler answer is to always fetch `development` and drop the ref logic — the examples change rarely enough that it would not lose much. I went the other way because it also closes the silent-wrong-content case.

@petersilva @andreleblanc11 @reidsunderland

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.