github-community-projects / github-community-projects/safe-settings

On pull request, check run `Safe-setting validator` is created but never runs ("Not working on a PR, returning...")

Open
#540 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
921
Forks
226
Avg merge
18h 3m
Merged PRs (30d)
14

Description

## Problem Description

I am seeing a non-deterministic issue with pull requests to the admin repo (we are using the fork model, so the pull requests come from a fork of the repo). Sometimes, the head commit on these pull requests has a pending (yellow) `Safe-setting validator` check run created with a description of `Queued — Waiting to run this check...`, but nothing more happens. The check run never completes.

### What is actually happening
The check run stays yellow/pending and never completes
### What is the expected behavior
The check run eventually completes.

### Error output, if available
With debug logging enabled, I see that safe-settings reacts to `pull_requests.opened` and creates a check run and then reacts to a `check_run.created` event.

However, the handling stops with error message `Not working on a PR, returning...` [here](https://github.com/github/safe-settings/blob/main-enterprise/index.js#L438-L439).

Looking at the Webhook Deliveries for my app, I indeed see that `check_suite.pull_requests` **is** empty (even though it was created by safe-settings. So it is not surprising that the check above is failing.

```
"check_suite": {
"id": 17454159573,
"node_id": "CS_kwDOKfMBec8AAAAEEFlW1Q",
"head_branch": null,
"head_sha": "23a5560b00614263576b69dacbb80c4dd73c6dce",
"status": "queued",
"conclusion": null,
"url": "https://api.github.com/repos/xxx/admin/check-suites/17454159573",
"before": null,
"after": null,
"pull_requests": [

],
```

At this point I am not sure if this is
* an issue with Github itself not populating `check_suite.pull_requests`, or whether
* safe-settings is making invalid assumptions about that field being populated or whether
* this is an artifact of me using a fork workflow? (During testing yesterday I did see the same problem though with a PR from a branch on the target repo).

## Context

### Are you using the hosted instance of probot/settings or running your own?
Our own.

### If running your own instance, are you using it with github.com or GitHub Enterprise?
github.com

#### Version of probot/settings
2.0.25

#### Version of GitHub Enterprise

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.