QuantEcon / QuantEcon/quantecon-book-theme

Bot commits from update-snapshots workflow do not trigger CI

Open
#358 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Jupyter Notebook
Stars
56
Forks
24
Avg merge
4h 33m
Merged PRs (30d)
4

Description

Problem

When /update-new-snapshots is triggered on a PR, the update-snapshots.yml workflow commits new visual baselines via the github-actions[bot] user using GITHUB_TOKEN. These bot commits do not trigger CI workflows due to two factors:

  1. The commit message contains [skip ci] — explicitly skipping CI.
  2. GitHub's infinite loop prevention: commits made with GITHUB_TOKEN do not trigger push or pull_request events, regardless of the commit message.

This means after new snapshots are committed, there's no automated CI run to validate the visual tests pass with the new baselines. You must manually close/reopen the PR to trigger CI.

Current Workaround

Close and reopen the PR after /update-new-snapshots commits the snapshots. This triggers pull_request events and runs CI.

Potential Solutions
  1. Use a PAT or GitHub App token instead of GITHUB_TOKEN in the update-snapshots workflow. Commits made with a PAT will trigger workflows. Requires adding a secret to the repo.
  2. Add a verification step to the update-snapshots workflow that runs the Playwright tests again after committing, to validate within the same workflow run.
  3. Add a workflow_dispatch trigger to ci.yml and have the update-snapshots workflow trigger it via the GitHub API after committing.
  4. Remove [skip ci] from the commit message — this alone won't fix it (GITHUB_TOKEN limitation), but would be needed alongside any of the above solutions.
Affected Workflows
  • .github/workflows/update-snapshots.yml — commits with [skip ci] and GITHUB_TOKEN
  • .github/workflows/ci.yml — triggers on push to main and pull_request

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .github/workflows/update-snapshots.yml and .github/workflows/ci.yml, then review how the snapshot commit uses GITHUB_TOKEN and how CI is triggered. Choose and implement a documented approach that automatically validates the updated visual baselines after the workflow commits them, and confirm the relevant Playwright tests run successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, playwright
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.