github-community-projects / github-community-projects/safe-settings

Can't get PR evaluation to run in dry-run mode (run via GitHub actions)

Open
#739 9 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
921
Forks
226
Avg merge
18h 3m
Merged PRs (30d)
14

Description

## Problem Description

We can't get safe-settings to run in dry-run (nop?) mode when evaluating pull requests on topic branch, run via GitHub actions. It always enforces the rules for both github triggers `pull_request` and `push` from main branch even though on topic branch.

Are we missing a setting?

We are on rc version `2.1.15-rc.1`, feel free to discard this issue if we are reporting it premature. We are ready to wait for future versions, I just wanted to report this back. Great work by the way! 👍

### What is actually happening

We see that the settings evaluated are not from the topic branch (they are from main branch) and that the debug statements display that it is not run in nop mode. Any changes are enforced from main branch even though it's evaluation intended.

It's as if it doesn't recognize that it should be run in dry-run (nop) mode and picks up the topic branch name.

### What is the expected behavior

That when run with event trigger is `pull_request` that the settings evaluated will be from the topic branch (not main branch) and that the changes on topic branch ARE NOT enforced, just gets output on the PR.

### Error output, if available

The log from full-sync.js when evaluating a pull request with `LOG_LEVEL: debug` displays that

a) The GET statement doesn't invoke the GitHub api for topic branch, but just main branch. I would expect that it tried to look the file from `?ref=topic-branch` or something like that

```log
DEBUG (github): GitHub request: GET https://api.github.com/repos/OurOrg/our-repo/contents/safe-settings%2Frepos%test-repo.yml - 200
```

b) The output from probat displays it's not run in nop:

```log
DEBUG (probot): Not run in nop
```

## Context

Running on GitHub Enterprise. Running safe-settings via GitHub Action. Run with pull_request trigger.

```yml
on:
# PR - Pull request trigger
pull_request:
```

Run with settings mentioned in [example file](https://github.com/github/safe-settings/blob/main-enterprise/docs/github-action.md#example-gha-workflow). Almost as example. The admin repo has a custom name.

```yml
SAFE_SETTINGS_VERSION: 2.1.15-rc.1

...

- run: npm run full-sync
working-directory: ${{env.SAFE_SETTINGS_CODE_DIR}}
env:
# Configuration files placed in safe-settings-folder in the ADMIN_REPO
ADMIN_REPO: our-admin-repo
CONFIG_PATH: safe-settings
SETTINGS_PATH: ${{github.workspace}}/safe-settings/settings.yml
DEPLOYMENT_CONFIG_FILE: ${{github.workspace}}/safe-settings/deployment-settings.yml

# GitHub App settings
GH_ORG: ${{vars.SAFE_SETTINGS_GH_ORG}}
APP_ID: ${{vars.SAFE_SETTINGS_APP_ID}}
PRIVATE_KEY: ${{secrets.SAFE_SETTINGS_PRIVATE_KEY}}
GITHUB_CLIENT_ID: ${{vars.SAFE_SETTINGS_GITHUB_CLIENT_ID}}
GITHUB_CLIENT_SECRET: ${{secrets.SAFE_SETTINGS_GITHUB_CLIENT_SECRET}}

# Logging
LOG_LEVEL: debug # info, trace, debug
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.