aws-amplify / aws-amplify/amplify-hosting

Frontend is deploying when tests fail

Open
#2,500 3 comments 0 reactions 0 assignees View on GitHub
e2e-cypress-tests ssr usability UX
Dominant language
Dockerfile
Stars
481
Forks
123
PR merge metrics
No merged PRs in 30d

Description

### Before opening, please confirm:

- [X] I have checked to see if my question is addressed in the [FAQ](https://github.com/aws-amplify/amplify-console/blob/master/FAQ.md).
- [X] I have [searched for duplicate or closed issues](https://github.com/aws-amplify/amplify-console/issues?q=is%3Aissue+).
- [X] I have read the guide for [submitting bug reports](https://github.com/aws-amplify/amplify-console/blob/master/CONTRIBUTING.md).
- [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

### App Id

d29mr6lvexeqms

### Region

us-east-1

### Amplify Console feature

E2E tests, Logging

### Describe the bug

if the E2E tests on my app fail, the app gets deployed regardless.
The UI will show the test has failed and the "deploy" stage hasn't started:
![unknown](https://user-images.githubusercontent.com/76787794/149006437-ad270a77-fd4b-4fc8-b824-e2ec413f5083.png)

But the site has been deployed, it has all the changes in the commit on the frontend, and the Next JS lambdas also get deployed.

I have since remade the entire amplify app, but still have the same problem.

### Expected behavior

Frontend doesnt deploy if the tests fail

### Reproduction steps

Unfortunately I'm unsure what causes this. I've tried removing certain resources but the problem was likely there when the tests were passing so I'm yet to find the cause.

### Build Settings

```yaml
version: 1
backend:
phases:
build:
commands:
- amplifyPush --simple
frontend:
phases:
preBuild:
commands:
- nvm install 16
- yarn install
- yarn run lint
build:
commands:
- yarn next build
artifacts:
baseDirectory: .next
files:
- '**/*'
cache:
paths:
- node_modules/**/*
- .next/cache/**/*
test:
phases:
preTest:
commands:
- nvm install 16
- yarn add mocha mochawesome mochawesome-merge mochawesome-report-generator
test:
commands:
- yarn start-test 'NEXT_PUBLIC_ISTEST=true next build && next start' 3000 'CYPRESS_TEST_USER=$TEST_USER CYPRESS_ENVIRONMENT=staging ELECTRON_ENABLE_LOGGING=1 yarn cypress run --reporter mochawesome --reporter-options "reportDir=cypress/report/mochawesome-report,overwrite=false,html=false,json=true,timestamp=mmddyyyy_HHMMss"'
postTest:
commands:
- yarn mochawesome-merge cypress/report/mochawesome-report/mochawesome*.json > cypress/report/mochawesome.json
artifacts:
baseDirectory: cypress
configFilePath: '**/mochawesome.json'
files:
- '**/*.png'
- '**/*.mp4'
cache:
paths:
- node_modules/**/*
- .next/cache/**/*
```
```

### Additional information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the provided Amplify build settings and the E2E test and deployment stages, then inspect the build logs for a failing Cypress command and the subsequent frontend or backend deployment. Reproduce the reported sequence if possible. Done means a failed E2E test prevents the frontend and related changes from being deployed, with the behavior reflected consistently in the UI and logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, cypress, javascript, nextjs
Domain
ci-cd, cloud, frontend, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.