cypress-io / cypress-io/code-coverage

Server-side coverage not reported when using next.js

Open
#557 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
445
Forks
115
PR merge metrics
No merged PRs in 30d

Description

Proof of concept repository: https://github.com/romandecker/next-ts-with-cypress-coverage

Steps to reproduce:
1. `npm install`
2. `npm run dev`
3. In another terminal: `DEBUG=code-coverage npm run cypress run`
4. `npm run see-cy-coverage` and verify that `getServerSideProps()` in `pages/index.tsx` was called and coverage was tracked successfully, but the function it calls, in `no-coverage-here/some-file.ts` still shows up as having 0% coverage, even though it was clearly executed.

**Logs and screenshots**

```
> cypress-coverage@1.0.0 cypress
> cypress "run"

Missing baseUrl in compilerOptions. tsconfig-paths will be skipped
code-coverage combined NYC options { 'report-dir': 'cypress-coverage', reporter: [ 'json', 'lcov' ], extension: [ '.js', '.jsx', '.ts', '.tsx' ], excludeAfterRemap: false, extends: './.nycrc.json', include: [ 'pages/**/*', 'no-coverage-here/**/*' ], exclude: [ 'pages/api/__coverage__.js' ], all: true, sourceMap: true, instrument: true } +0ms

====================================================================================================

(Run Starting)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 5.3.0 │
│ Browser: Electron 83 (headless) │
│ Node Version: v16.14.0 (/Users/romande/.nvm/versions/node/v16.14.0/bin/node) │
│ Specs: 2 found (about.spec.ts, index.spec.ts) │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────────────────────────────────────────────

Running: about.spec.ts (1 of 2)

app
code-coverage parsed sent coverage +0ms
code-coverage wrote coverage file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json +7ms
code-coverage parsed sent coverage +174ms
code-coverage wrote coverage file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json +3ms
code-coverage NYC file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json has 3 key(s) +6s
code-coverage 1 key /Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx file path /Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx +0ms
code-coverage 2 key /Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx file path /Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx +0ms
code-coverage 3 key /Users/romande/projects/next-ts-with-cypress-coverage/pages/index.tsx file path /Users/romande/projects/next-ts-with-cypress-coverage/pages/index.tsx +0ms
code-coverage in file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json all files are not found? false +1ms
code-coverage NYC file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json has 3 key(s) +0ms
code-coverage nyc needs to report on all included files +18ms
code-coverage include all files options: { all: true, include: [ 'pages/**/*', 'no-coverage-here/**/*' ], exclude: [ 'pages/api/__coverage__.js' ], extension: [ '.js', '.jsx', '.ts', '.tsx' ] } +1ms
code-coverage searching files to include using patterns [ 'pages/**/*', 'no-coverage-here/**/*', '!pages/api/__coverage__.js', '!**/node_modules/**' ] +0ms
code-coverage found 4 file(s) +10ms
/Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/pages/index.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/no-coverage-here/some-file.ts
code-coverage coverage has 3 record(s) +9ms
coverage has the following first paths
/Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/pages/index.tsx
code-coverage adding empty coverage for file /Users/romande/projects/next-ts-with-cypress-coverage/no-coverage-here/some-file.ts +0ms
code-coverage includeAllFiles saving updated file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json +0ms
code-coverage there are 4 keys in the file +0ms
code-coverage calling NYC reporter with options { 'report-dir': '/Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage', reporter: [ 'json', 'lcov' ], extension: [ '.js', '.jsx', '.ts', '.tsx' ], excludeAfterRemap: false, extends: './.nycrc.json', include: [ 'pages/**/*', 'no-coverage-here/**/*' ], exclude: [ 'pages/api/__coverage__.js' ], all: true, sourceMap: true, instrument: true, 'temp-dir': '/Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output', tempDir: '/Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output', reportDir: '/Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage' } +22ms
code-coverage current working directory is /Users/romande/projects/next-ts-with-cypress-coverage +1ms
code-coverage after reporting, returning the report folder name /Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage +988ms
code-coverage Final coverage in /Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage/coverage-final.json +0ms
code-coverage There are 4 key(s) in /Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage/coverage-final.json +1ms
code-coverage ❓ /Users/romande/projects/next-ts-with-cypress-coverage/no-coverage-here/some-file.ts statements covered 0/0 +0ms
code-coverage ✅ /Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx statements covered 1/1 +0ms
code-coverage ✅ /Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx statements covered 8/8 +0ms
code-coverage ✅ /Users/romande/projects/next-ts-with-cypress-coverage/pag ✓ redirects to / (2070ms)

1 passing (3s)

es/index.tsx statements covered 3/3 +0ms

(Results)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 1 │
│ Passing: 1 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: true │
│ Duration: 3 seconds │
│ Spec Ran: about.spec.ts │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

(Video)

- Started processing: Compressing to 32 CRF
- Finished processing: /Users/romande/projects/next-ts-with-cypress-coverage/cypre (0 seconds)
ss/videos/about.spec.ts.mp4

────────────────────────────────────────────────────────────────────────────────────────────────────

Running: index.spec.ts (2 of 2)

landing
code-coverage parsed sent coverage +4s
code-coverage wrote coverage file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json +2ms
code-coverage parsed sent coverage +31ms
code-coverage wrote coverage file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json +1ms
code-coverage NYC file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json has 3 key(s) +5s
code-coverage 1 key /Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx file path /Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx +0ms
code-coverage 2 key /Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx file path /Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx +0ms
code-coverage 3 key /Users/romande/projects/next-ts-with-cypress-coverage/pages/index.tsx file path /Users/romande/projects/next-ts-with-cypress-coverage/pages/index.tsx +0ms
code-coverage in file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json all files are not found? false +1ms
code-coverage NYC file /Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output/out.json has 3 key(s) +0ms
code-coverage nyc needs to report on all included files +17ms
code-coverage include all files options: { all: true, include: [ 'pages/**/*', 'no-coverage-here/**/*' ], exclude: [ 'pages/api/__coverage__.js' ], extension: [ '.js', '.jsx', '.ts', '.tsx' ] } +0ms
code-coverage searching files to include using patterns [ 'pages/**/*', 'no-coverage-here/**/*', '!pages/api/__coverage__.js', '!**/node_modules/**' ] +1ms
code-coverage found 4 file(s) +18ms
/Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/pages/index.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/no-coverage-here/some-file.ts
code-coverage coverage has 3 record(s) +1ms
coverage has the following first paths
/Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx
/Users/romande/projects/next-ts-with-cypress-coverage/pages/index.tsx
code-coverage adding empty coverage for file /Users/romande/projects/next-ts-with-cypress-coverage/no-coverage-here/some-file.ts +0ms
code-coverage includeAllFiles saving updated file /Users/romande/projects/next-t ✓ should render (753ms)

1 passing (888ms)

s-with-cypress-coverage/.nyc_output/out.json +0ms
code-coverage there are 4 keys in the file +0ms
code-coverage calling NYC reporter with options { 'report-dir': '/Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage', reporter: [ 'json', 'lcov' ], extension: [ '.js', '.jsx', '.ts', '.tsx' ], excludeAfterRemap: false, extends: './.nycrc.json', include: [ 'pages/**/*', 'no-coverage-here/**/*' ], exclude: [ 'pages/api/__coverage__.js' ], all: true, sourceMap: true, instrument: true, 'temp-dir': '/Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output', tempDir: '/Users/romande/projects/next-ts-with-cypress-coverage/.nyc_output', reportDir: '/Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage' } +20ms
code-coverage current working directory is /Users/romande/projects/next-ts-with-cypress-coverage +0ms
code-coverage after reporting, returning the report folder name /Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage +19ms
code-coverage Final coverage in /Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage/coverage-final.json +0ms
code-coverage There are 4 key(s) in /Users/romande/projects/next-ts-with-cypress-coverage/cypress-coverage/coverage-final.json +0ms
code-coverage ❓ /Users/romande/projects/next-ts-with-cypress-coverage/no-coverage-here/some-file.ts statements covered 0/0 +0ms
code-coverage ✅ /Users/romande/projects/next-ts-with-cypress-coverage/pages/_app.tsx statements covered 1/1 +0ms
code-coverage ✅ /Users/romande/projects/next-ts-with-cypress-coverage/pages/about.tsx statements covered 8/8 +0ms
code-coverage ✅ /Users/romande/projects/next-ts-with-cypress-coverage/pages/index.tsx statements covered 3/3 +0ms

(Results)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 1 │
│ Passing: 1 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: true │
│ Duration: 0 seconds │
│ Spec Ran: index.spec.ts │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

(Video)

- Started processing: Compressing to 32 CRF
- Finished processing: /Users/romande/projects/next-ts-with-cypress-coverage/cypre (0 seconds)
ss/videos/index.spec.ts.mp4

====================================================================================================

(Run Finished)

Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ about.spec.ts 00:03 1 1 - - - │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ index.spec.ts 885ms 1 1 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 00:04 2 2 - - -

```

![image](https://user-images.githubusercontent.com/439587/165535329-1027d9e4-ea4a-4e97-bb47-c7a5268a993b.png)

**Versions**

- What is this plugin's version? 3.9.12
- What is Cypress version? 9.6.0
- What is your operating system? Mac OS
- What is the shell? zsh
- What is the Node version? v16.14.0
- What is the NPM version? v8.5.0
- How do you instrument your application? istanbul via the babel-plugin, see the example repo
- When running tests, if you open the web application in regular browser, and open DevTools, do you see `window.__coverage__` object? yes
- Is there `.nyc_output` folder? Is there `.nyc_output/out.json` file. Is it empty? Can you paste at least part of it so we can see the keys and file paths? yes, it just doesn't cover certainly included files
- Do you run Cypress tests in a Docker container? no

**Describe the bug**

Coverage is being measured correctly, but ONLY for the `pages` folder. Anything outside the pages folder (e.g. `no-coverage-here`) always remains uncovered, despite being called.

**Link to the repo**
https://github.com/romandecker/next-ts-with-cypress-coverage

Originally found this issue here: https://github.com/kylemh/next-ts-with-cypress-coverage/issues/3

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.