cypress-io / cypress-io/code-coverage

@cypress/code-coverage attempts to collect back-end coverage when we are not running end-to-end tests

Open
#473 3 comments 2 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
445
Forks
115
PR merge metrics
No merged PRs in 30d

Description

**Logs and screenshots**
```
code-coverage combined NYC options { 'report-dir': './coverage', reporter: [ 'lcov', 'json-summary', 'text', 'cobertura' ], extension: [ '.jsx', '.js' ], excludeAfterRemap: false, sourceMap: false, instrument: false } +0ms
code-coverage reset code coverage in interactive mode +0ms
code-coverage reset code coverage in interactive mode +2s
code-coverage parsed sent coverage +8s
code-coverage wrote coverage file /Users/nyee2/dev/plugins/campaign-manager-web-plugin/.nyc_output/out.json +30ms
code-coverage parsed sent coverage +721ms
code-coverage wrote coverage file /Users/nyee2/dev/plugins/campaign-manager-web-plugin/.nyc_output/
code-coverage parsed sent coverage +2s
code-coverage wrote coverage file /Users/nyee2/dev/plugins/campaign-manager-web-plugin/.nyc_output/out.json +61ms
code-coverage parsed sent coverage +915ms
code-coverage wrote coverage file /Users/nyee2/dev/plugins/campaign-manager-web-plugin/.nyc_output/out.json +54ms
code-coverage parsed sent coverage +799ms
code-coverage wrote coverage file /Users/nyee2/dev/plugins/campaign-manager-web-plugin/.nyc_output/out.json +57ms
code-coverage parsed sent coverage +3s
code-coverage wrote coverage file /Users/nyee2/dev/plugins/campaign-manager-web-plugin/.nyc_output/out.json +50ms
code-coverage parsed sent coverage +2s
code-coverage wrote coverage file /Users/nyee2/dev/plugins/campaign-manager-web-plugin/.nyc_output/out.json +53ms
```

**Versions**

- @cypress/code-coverage **v3.9.7**
- Cypress **7.0.1**
- OS: **MacOS Big Sur 11.4**
- shell: **bash**
- Node **v14.16.0**
- NPM **6.14.11**
- How do you instrument your application? **`babel-plugin-istanbul`**
- do you see `window.__coverage__` object? **yes, `window.__coverage__` exists and instrumentation works**
- Is there `.nyc_output` folder? **yes coverage data is generated**
- custom NYC settings in `package.json`
```
"nyc": {
"extension": [
".jsx",
".js"
],
"reporter": [
"lcov",
"json-summary",
"text",
"cobertura"
],
"sourceMap": false,
"instrument": false
},
```
- Do you run Cypress tests in a Docker container? **yes in CI, MacOS locally**

**Describe the bug**
The logic to determine if cypress is running an "end-to-end" test incorrectly resolves as `true` in my cypress tests, as our `baseUrl` and `proxyUrl` are different.

Code reference in question:
https://github.com/cypress-io/code-coverage/blob/c51751f9f9875c88aaacf648c738bec63b4163f1/support.js#L168-L172

From the above code, `runningEndToEndTests` evaluates to true if your `baseUrl` and `proxyUrl` are different, which they are in my case but I am **_not_** running end-to-end tests.

![Cursor_and_campaign-manager-web-plugin](https://user-images.githubusercontent.com/155019/123369945-9a144480-d533-11eb-8224-a5dd72d85bf3.png)

As my `specType` is integration, after tests complete an attempt is made to reach a `/coverage` URL which does not exist, thus failing the test.
![Cursor_and_campaign-manager-web-plugin](https://user-images.githubusercontent.com/155019/123420014-cea5f180-d56f-11eb-8c29-eb363a317d6a.png)

I would like to propose perhaps setting & checking against a new config `codeCoverage.expectFrontendCoverageOnly` explicitly to determine if backend coverage needs to be collected (similar to the pre-existing `expectBackendCoverageOnly` config), eg. the above code reference could be augmented to preserve the existing behavior but allowing an explicit check against `expectFrontendCoverageOnly`

```
const runningEndToEndTests = baseUrl !== Cypress.config('proxyUrl')
const specType = Cypress._.get(Cypress.spec, 'specType', 'integration')
const isIntegrationSpec = specType === 'integration'
const expectFrontendCoverageOnly = Cypress._.get(Cypress.env('codeCoverage'), 'expectFrontendCoverageOnly',
false)

if (!expectFrontendCoverageOnly && runningEndToEndTests && isIntegrationSpec) {
```

**Link to the repo**
The code I am using @cypress/code-coverage is behind a private GH instance so I unfortunately can't share here. I have created https://github.com/cypress-io/code-coverage/pull/474 with the above suggestion

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.