cypress-io / cypress-io/code-coverage

`isTestFile` filter fails for window.__coverage__ object with Symbol key

Open
#908 0 comments 3 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
445
Forks
115
PR merge metrics
No merged PRs in 30d

Description

**Logs**

```
code-coverage combined NYC options { 'report-dir': './coverage', reporter: [ 'lcov', 'clover', 'json', 'json-summary' ], extension: [ '.js', '.cjs', '.mjs', '.ts', '.tsx', '.jsx' ], excludeAfterRemap: false } +0ms
code-coverage parsed sent coverage +0ms
code-coverage NYC file /Users/user/frontend/apps/my-app-tests/.nyc_output/out.json has 329 key(s) +5s
code-coverage 1 key /Users/user/frontend/apps/my-app/src/app/app.component.ts file path /Users/user/frontend/apps/my-app/src/app/app.component.ts +0ms
code-coverage 2 key /Users/user/frontend/libs/shared/util/core/src/lib/interfaces/base-query.interface.ts file path /Users/user/frontend/libs/shared/util/core/src/lib/interfaces/base-query.interface.ts +0ms
code-coverage 3 key /Users/user/frontend/libs/shared/util/core/src/lib/interfaces/base-record.interface.ts file path /Users/user/frontend/libs/shared/util/core/src/lib/interfaces/base-record.interface.ts +1ms
code-coverage in file /Users/user/frontend/apps/my-app-tests/.nyc_output/out.json all files are not found? false +98ms
code-coverage NYC file /Users/user/frontend/apps/my-app-tests/.nyc_output/out.json has 329 key(s) +62ms
code-coverage calling NYC reporter with options { 'report-dir': '/Users/user/frontend/apps/my-app-tests/coverage', reporter: [ 'lcov', 'clover', 'json', 'json-summary' ], extension: [ '.js', '.cjs', '.mjs', '.ts', '.tsx', '.jsx' ], excludeAfterRemap: false, 'temp-dir': '/Users/user/frontend/apps/my-app-tests/.nyc_output', tempDir: '/Users/user/frontend/apps/my-app-tests/.nyc_output', reportDir: '/Users/user/frontend/apps/my-app-tests/coverage' } +373ms
code-coverage current working directory is /Users/user/frontend/apps/my-app-tests +0ms
code-coverage after reporting, returning the report folder name /Users/user/frontend/apps/my-app-tests/coverage +786ms
code-coverage Final coverage in /Users/user/frontend/apps/my-app-tests/coverage/coverage-final.json +0ms
code-coverage There are 413 key(s) in /Users/user/frontend/apps/my-app-tests/coverage/coverage-final.json +34ms
```

**Versions**

- _What is this plugin's version?_ `3.13.6`
- _What is the Cypress version?_ `13.15.2`
- _What is the Node version?_ `22.4.1`
- _What is the NPM version?_ `10.8.1`
- _How do you instrument your application?_ with a loader `loader: '@skyux-sdk/istanbul-instrumenter-loader',` inside webpack config
- _When running tests, if you open the web application in a regular browser and open DevTools, do you see `window.__coverage__` object?_ yes
- _Is there a `.nyc_output` folder? Is there a `.nyc_output/out.json` file?_ yes, not empty
- _Do you have any custom NYC settings in `package.json` (`nyc` object) or in other [NYC config files](https://github.com/istanbuljs/nyc#configuration-files)?_ no
- _Do you run Cypress tests in a Docker container?_ no

**Describe the bug**
My app dynamically loads a wasm lib from assets at runtime:
```
const wasmFullUrl = joinParts(this._assetsUrl, 'my-wasm-lib/main.js');
const { init } = await import(/* webpackIgnore: true */ wasmFullUrl) as { init: () => Promise };
const initResult = await init();
```
Once the cypress test passes the import line, inside the `window.__coverage__` object, under Prototype, there is this key-value pair: `Symbol(wasm type): 0`.

![image](https://github.com/user-attachments/assets/d42d673a-ac99-4ce2-a646-ca8dd7efcc11)

After the test is run, while code coverage is processed, the `window.__coverage__` object reaches this filter:
```
const coverage = Cypress._.omitBy(totalCoverage, isTestFile)
```
and apparently the lodash omitBy method will also go through key,value pairs inside Prototype, which makes the isTestFile method be called with the Symbol(wasm type) key, causing the following line to fail (where filePath is Symbol):
```
// inside isTestFile
const filename = filePath.replace(workingDir, '')
```
And eventually the test fails with:
```
TypeError: filePath.replace is not a function
Because this error occurred during a `after each` hook we are skipping all of the remaining tests.
at isTestFile (webpack:///../../node_modules/@cypress/code-coverage/support-utils.js:31:0)
at filterSpecsFromCoverage (webpack:///../../node_modules/@cypress/code-coverage/support-utils.js:41:0)
at filterFilesFromCoverage (webpack:///../../node_modules/@cypress/code-coverage/support-utils.js:12:0)
at sendCoverage (webpack:///../../node_modules/@cypress/code-coverage/support.js:20:0)
at eval (webpack:///../../node_modules/@cypress/code-coverage/support.js:146:0)
at Array.forEach ()
at Context.eval (webpack:///../../node_modules/@cypress/code-coverage/support.js:145:0)
```

**Workaround**
```js
// delete the symbol from the coverage prototype after each test, before it is processed by cypress/code-coverage
let windowObj = {};
const saveWindowCoverageObject = (win: any) => windowObj = win;
beforeEach(() => {
cy.on('window:load', saveWindowCoverageObject);
});
afterEach(() => {
let symbolKey;
// eslint-disable-next-line no-underscore-dangle
Cypress._.omitBy(windowObj.__coverage__, (_, key) => {
if (typeof key === 'symbol') {
symbolKey = key;
}
return typeof key === 'symbol';
});
// eslint-disable-next-line no-underscore-dangle
delete windowObj.__coverage__.__proto__[symbolKey];
});
```

I tried to exclude the assets and the file that does the dynamic import from instrumentation, but the Symbol is still persisted inside the coverage object. I couldn't identify what actually adds the symbol or why.

It would be helpful if the post processing done inside `filterSpecsFromCoverage` could ignore the symbol keys in order to avoid this kind of failure.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.