cypress-io / cypress-io/github-action

Re-run only failed tests (jobs) for Cloud parallel recording

Open
#1,290 8 comments 10 reactions 0 assignees View on GitHub
Cypress Cloud type: feature
Dominant language
JavaScript
Stars
1.5k
Forks
353
Avg merge
1d 22h
Merged PRs (30d)
27

Description

- follows on from and replaces https://github.com/cypress-io/github-action/issues/531

## What would you like?

### Scenario

A GitHub Actions workflow contains a job using the [parallel](https://github.com/cypress-io/github-action/blob/master/README.md#parallel) option of `cypress-io/github-action` recording to [Cypress Cloud](https://cloud.cypress.io/) which runs Cypress tests in multiple containers.

One or more Cypress tests fail.

GitHub Actions offers the option [Re-run failed jobs](https://docs.github.com/en/actions/managing-workflow-runs/re-running-workflows-and-jobs#about-re-running-workflows-and-jobs)

![image](https://github.com/cypress-io/github-action/assets/66998419/e70ac7d4-7314-4401-a3ce-29df140bde37)

### Enhancement

GitHub Actions and Cypress Cloud should work together so that selecting [Re-run failed jobs](https://docs.github.com/en/actions/managing-workflow-runs/re-running-workflows-and-jobs#about-re-running-workflows-and-jobs) only re-runs failed jobs using a frozen set of load-balanced Cypress test specs, so that each failed job runs with the same test specs it used in its previous run. The Cypress tests which were distributed to other jobs, and which were previously successful, should not be re-run.

## Why is this needed?

Reduce the overhead of re-running flaky tests without also running tests which were previously successful.

## Other

The [README > parallel](https://github.com/cypress-io/github-action/blob/master/README.md#parallel) documentation section currently states:

> If you use the GitHub Actions facility for [Re-running workflows and jobs](https://docs.github.com/en/actions/managing-workflow-runs/re-running-workflows-and-jobs), note that [Re-running failed jobs in a workflow](https://docs.github.com/en/actions/managing-workflow-runs/re-running-workflows-and-jobs?tool=webui#re-running-failed-jobs-in-a-workflow) is not suited for use with parallel recording into Cypress Cloud. Re-running failed jobs in this situation does not simply re-run failed Cypress tests. Instead it re-runs **all** Cypress tests, load-balanced over the containers with failed jobs.
>
> To optimize runs when there are failing tests present, refer to optional [Cypress Cloud Smart Orchestration](https://docs.cypress.io/cloud/features/smart-orchestration/overview/) Premium features:
> - [Spec Prioritization](https://docs.cypress.io/cloud/features/smart-orchestration/spec-prioritization)
> - [Auto Cancellation](https://docs.cypress.io/guides/cloud/smart-orchestration/run-cancellation). See also [Specify auto cancel after failures](https://github.com/cypress-io/github-action/blob/master/README.md#specify-auto-cancel-after-failures) for details of how to set this option in a Cypress GH Action workflow.

- See previous issue #531 for extensive historical and background reading on this issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.