Generating code coverage on Azure pipelines

Open
#19 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
azure, playwright, typescript, vite
Domain
ci-cd, cloud, testing-qa

Research direction

Start with vite.config.ts and baseFixtures.ts, then reproduce the listed Playwright and nyc commands in the Azure pipeline environment. Compare the local and pipeline coverage-output behavior; done means JSON coverage files are created under the generated output directory and npx nyc report --reporter=text produces a report.

Written by the indexing model from the issue text.

Description

System info

● Playwright Version: Latest
● Operating System: Win 11
● Browser: MS Edge

Steps
npm i && npx playwright install --with-deps
npm i vite-plugin-istanbul
npm i nyc
npx playwright test --project "e2e tests on pipeline"
npx nyc report --reporter=text

Configs:
vite.config.ts

image

baseFixtures.ts

`import * as fs from 'fs'
import * as path from 'path'
import * as crypto from 'crypto'
import { fileURLToPath } from 'url'
import { test as baseTest } from '@playwright/test'

const filename = fileURLToPath(import.meta.url)
const __dirname = path.dirname(filename)
const istanbulCLIOutput = path.join(__dirname, '..', '.nyc_output')

export function generateUUID(): string {
return crypto.randomBytes(16).toString('hex')
}

export const test = baseTest.extend({
context: async ({ context }, use) => {
await context.addInitScript(() =>
window.addEventListener('beforeunload', () =>
(window as any).collectIstanbulCoverage(JSON.stringify((window as any).coverage)),
),
)
await fs.promises.mkdir(istanbulCLIOutput, { recursive: true })
await context.exposeFunction('collectIstanbulCoverage', (coverageJSON: string) => {
if (coverageJSON)
fs.writeFileSync(path.join(istanbulCLIOutput, playwright_coverage_${generateUUID()}.json), coverageJSON)
})
await use(context)
for (const page of context.pages()) {
await page.evaluate(() => (window as any).collectIstanbulCoverage(JSON.stringify((window as any).coverage)))
}
},
})

export const expect = test.expect`

Expected
When running the tests locally, the coverage json files are generated under the nyc_report folder as expected and the report can be displayed normally either in text or html form.

On the azure pipeline environment, all tests are still passing, the nyc_folder is created but there are no json files created under it.
Thus, this is the output I get from the npx nyc report --reporter=text command.

image

I am not really sure what I have done incorrectly or whether this tool was never supposed to be able to work on azure pipelines at all though, Some input or tips towards the right direction would be greatly appreciated. Thank you.

Dominant language
TypeScript
Stars
143
Forks
20
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from mxschmitt/playwright-test-coverage

All issues in mxschmitt/playwright-test-coverage

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.