cypress-io / cypress-io/code-coverage

Not working with Storybook 7

Open
#657 7 comments 1 reaction 0 assignees View on GitHub
E2E triaged
Dominant language
TypeScript
Stars
445
Forks
115
PR merge metrics
No merged PRs in 30d

Description

**Logs and screenshots**

click for logs...

```bash
[storybook-with-cypress] yarn test
yarn run v1.22.17
$ DEBUG=code-coverage cypress run
[62033:0519/165728.394350:ERROR:node_bindings.cc(279)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[62033:0519/165728.394372:ERROR:node_bindings.cc(279)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[62033:0519/165728.394376:ERROR:node_bindings.cc(279)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.

DevTools listening on ws://127.0.0.1:54544/devtools/browser/b64f05d0-47fb-4b88-a3c7-35082697cdab
code-coverage combined NYC options { 'report-dir': './coverage', reporter: [ 'lcov', 'clover', 'json', 'json-summary' ], extension: [ '.js', '.cjs', '.mjs', '.ts', '.tsx', '.jsx' ], excludeAfterRemap: false, all: true, include: [ 'stories/**/*' ] } +0ms

====================================================================================================

(Run Starting)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 12.12.0 │
│ Browser: Electron 106 (headless) │
│ Node Version: v18.15.0 (/Users/jfairley/n/bin/node) │
│ Specs: 1 found (button.cy.js) │
│ Searched: cypress/e2e/**/*.cy.{js,jsx,ts,tsx} │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────────────────────────────────────────────

Running: button.cy.js (1 of 1)

Button
code-coverage NYC file /Users/jfairley/dev/code/storybook-with-cypress/.nyc_output/out.json has 10 key(s) +3s
code-coverage 1 key /Users/jfairley/dev/code/storybook-with-cypress/cypress/support/commands.js file path /Users/jfairley/dev/code/storybook-with-cypress/cypress/support/commands.js +0ms
code-coverage 2 key /Users/jfairley/dev/code/storybook-with-cypress/cypress/support/e2e.js file path /Users/jfairley/dev/code/storybook-with-cypress/cypress/support/e2e.js +0ms
code-coverage 3 key /Users/jfairley/dev/code/storybook-with-cypress/cypress.config.js file path /Users/jfairley/dev/code/storybook-with-cypress/cypress.config.js +0ms
code-coverage in file /Users/jfairley/dev/code/storybook-with-cypress/.nyc_output/out.json all files are not found? false +0ms
code-coverage NYC file /Users/jfairley/dev/code/storybook-with-cypress/.nyc_output/out.json has 10 key(s) +0ms
code-coverage nyc needs to report on all included files +0ms
code-coverage include all files options: { all: true, include: [ 'stories/**/*' ], exclude: undefined, extension: [ '.js', '.cjs', '.mjs', '.ts', '.tsx', '.jsx' ] } +1ms
code-coverage searching files to include using patterns [ 'stories/**/*', '!**/node_modules/**' ] +0ms
code-coverage found 3 file(s) +4ms
/Users/jfairley/dev/code/storybook-with-cypress/stories/Button.jsx
/Users/jfairley/dev/code/storybook-with-cypress/stories/Button.stories.js
/Users/jfairley/dev/code/storybook-with-cypress/stories/button.css
code-coverage coverage has 10 record(s) +1ms
coverage has the following first paths
/Users/jfairley/dev/code/storybook-with-cypress/cypress/support/commands.js
/Users/jfairley/dev/code/storybook-with-cypress/cypress/support/e2e.js
/Users/jfairley/dev/code/storybook-with-cypress/cypress.config.js
/Users/jfairley/dev/code/storybook-with-cypress/stories/Button.stories.js
code-coverage calling NYC reporter with options { 'report-dir': '/Users/jfairley/dev/code/storybook-with-cypress/coverage', reporter: [ 'lcov', 'clover', 'json', 'json-summary' ], extension: [ '.js', '.cjs', '.mjs', '.ts', '.tsx', '.jsx' ], excludeAfterRemap: false, all: true, include: [ 'stories/**/*' ], 'temp-dir': '/Users/jfairley/dev/code/storybook-with-cypress/.nyc_output', tempDir: '/Users/jfairley/dev/code/storybook-with-cypress/.nyc_output', reportDir: '/Users/jfairley/dev/code/storybook-with-cypress/coverage' } +6ms
code-coverage current working directory is /Users/jfairley/dev/code/storybook-with-cypress +0ms
code-coverage after reporting, returning the report folder name /Users/jfairley/dev/code/storybook-with-cypress/coverage +51ms
code-coverage Final coverage in /Users/jfairley/dev/code/storybook-with-cypress/coverage/coverage-final.json +0ms
code-coverage There are 10 key(s) in /Users/jfairley/dev/code/storybook-with-cypress/coverage/coverage-final.json +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/cypress.config.js statements covered 0/0 +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/coverage/lcov-report/block-navigation.js statements covered 0/0 +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/coverage/lcov-report/prettify.js statements covered 0/0 +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/coverage/lcov-report/sorter.js statements covered 0/0 +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/cypress/e2e/button.cy.js statements covered 0/0 +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/cypress/support/commands.js statements covered 0/0 +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/cypress/support/e2e.js statements covered 0/0 +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/stories/Button.jsx statements covered 0/0 +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/stories/Button.stories.js statements covered 0/0 +0ms
code-coverage ❓ /Users/jfairley/dev/code/storybook-with-cypress/stories/button.css statements covered 0/0 +0ms
✓ should render (329ms)

1 passing (1s)

(Results)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 1 │
│ Passing: 1 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: true │
│ Duration: 1 second │
│ Spec Ran: button.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

(Video)

- Started processing: Compressing to 32 CRF
- Finished processing: 0 seconds

- Video output: /Users/jfairley/dev/code/storybook-with-cypress/cypress/videos/button.cy.js.mp4

====================================================================================================

(Run Finished)

Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ button.cy.js 00:01 1 1 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 00:01 1 1 - - -

✨ Done in 6.19s.
```

**Versions**

- What is this plugin's version? If this is NOT the latest [released version](https://github.com/cypress-io/code-coverage/releases) can you try the latest version, please?
- 3.10.4
- If the plugin worked before in version X, but stopped after upgrading to version Y, please try the [released versions](https://github.com/cypress-io/code-coverage/releases) between X and Y to see where the breaking change was.
- The break occurs when upgrading storybook to v7.
- What is Cypress version?
- 12.12.0
- What is your operating system?
- MacOS Ventura 13.1
- What is the shell?
- zsh
- What is the Node version?
- 18.15.0
- What is the NPM version?
- 9.5.0
- How do you instrument your application? Cypress [does not instrument web application code](https://github.com/cypress-io/code-coverage#instrument-your-application), so you need to do it yourself.
- I have two methods applied, both of which I suspect would work without the other, but I'm trying to cover my bases.
- `"@storybook/addon-coverage"`
- `"babel-plugin-istanbul"` registered in `.babelrc.json`
- When running tests, if you open the web application in regular browser, and open DevTools, do you see `window.__coverage__` object? Can you paste a screenshot?
- yes
- Screenshot 2023-05-19 at 4 56 40 PM
- Is there `.nyc_output` folder? Is there `.nyc_output/out.json` file. Is it empty? Can you paste at least part of it so we can see the keys and file paths?
- Not fully empty, but no coverage reported in it.
-
see `.nyc_output/out.json`

```json
{
"/Users/jfairley/dev/code/storybook-with-cypress/cypress/support/commands.js": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/cypress/support/commands.js",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {},
"_coverageSchema": "1a1c01bbd47fc00a2c39e90264f33305004495a9",
"hash": "b100d0ceb86dc20ac064a45298efa0bdb436b8da"
},
"/Users/jfairley/dev/code/storybook-with-cypress/cypress/support/e2e.js": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/cypress/support/e2e.js",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {},
"_coverageSchema": "1a1c01bbd47fc00a2c39e90264f33305004495a9",
"hash": "3c1c42d7c6490def901876d78c647964f3422f2a"
},
"/Users/jfairley/dev/code/storybook-with-cypress/cypress.config.js": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/cypress.config.js",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {}
},
"/Users/jfairley/dev/code/storybook-with-cypress/stories/Button.stories.js": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/stories/Button.stories.js",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {}
},
"/Users/jfairley/dev/code/storybook-with-cypress/coverage/lcov-report/block-navigation.js": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/coverage/lcov-report/block-navigation.js",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {}
},
"/Users/jfairley/dev/code/storybook-with-cypress/coverage/lcov-report/prettify.js": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/coverage/lcov-report/prettify.js",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {}
},
"/Users/jfairley/dev/code/storybook-with-cypress/coverage/lcov-report/sorter.js": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/coverage/lcov-report/sorter.js",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {}
},
"/Users/jfairley/dev/code/storybook-with-cypress/cypress/e2e/button.cy.js": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/cypress/e2e/button.cy.js",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {}
},
"/Users/jfairley/dev/code/storybook-with-cypress/stories/Button.jsx": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/stories/Button.jsx",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {}
},
"/Users/jfairley/dev/code/storybook-with-cypress/stories/button.css": {
"path": "/Users/jfairley/dev/code/storybook-with-cypress/stories/button.css",
"statementMap": {},
"fnMap": {},
"branchMap": {},
"s": {},
"f": {},
"b": {}
}
}
```

- Do you have any custom NYC settings in `package.json` (`nyc` object) or in other [NYC config files](https://github.com/istanbuljs/nyc#configuration-files)
-
open `.nycrc`

```json
{
"all": true,
"include": ["stories/**/*"]
}
```

- Do you run Cypress tests in a Docker container?
- no

**Describe the bug**

I run Cypress e2e tests against my instrumented Storybook stories for React.

After upgrading to Storybook 7, when I run Cypress, I see `window.__coverage__` as expected, but `.nyc_output/out.json` never shows coverage within it.

This worked with Storybook 6, but only upgrading to Storybook 7 breaks cypress coverage.

I assume the problem might be with conflicting transitive dependencies, but I don't know.

**Link to the repo**

I have created a new repository to showcase the issue. My application repositories are private.

https://github.com/jfairley/storybook-with-cypress

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.