GoogleChrome / GoogleChrome/lighthouse-ci

Improved assertMatrix overlap behavior

Open
#511 7 comments 0 reactions 0 assignees View on GitHub
enhancement P1
Dominant language
JavaScript
Stars
7.1k
Forks
716
PR merge metrics
No merged PRs in 30d

Description

I was trying to use assertMatrix with preset like this:

**myconfig.yml**

```
assert:
preset: "lighthouse:recommended"
assertMatrix:
- matchingUrlPattern: "devserver.com"
assertions:
is-crawlable:
- off # Dev servers are not crawlable.
```

However, this results in an error:

```
Error: Cannot use assertMatrix with other options
at getAllAssertionResults (/usr/local/lib/node_modules/@lhci/cli/node_modules/@lhci/utils/src/assertions.js:486:13)
at Object.runCommand (/usr/local/lib/node_modules/@lhci/cli/src/assert/assert.js:58:22)
at run (/usr/local/lib/node_modules/@lhci/cli/src/cli.js:106:23)
at Object. (/usr/local/lib/node_modules/@lhci/cli/src/cli.js:137:1)
at Module._compile (internal/modules/cjs/loader.js:1063:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
at Module.load (internal/modules/cjs/loader.js:928:32)
at Function.Module._load (internal/modules/cjs/loader.js:769:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
at internal/main/run_main_module.js:17:47

```

What I'm trying to do: I want to use the default checks, but I have a few specific pages that I know will fail a few specific checks. So I was hoping to use `assertMatrix` to disable such checks for those specific pages.

If I remove `preset`, I can use `assertMatrix`, but then I have to create the entire list of assertions that `"lighthouse-recommended"` checks for myself. In that case, it would be easier to create several different config files for each case that needs special exceptions... but I was hoping to avoid creating multiple config files by using `assertMatrix`. Is there some way to do that?

Contributor guide

Open the contributing guide

Research direction

Start in utils/src/assertions.js at getAllAssertionResults, then follow the assert command path shown in src/assert/assert.js to understand how preset and assertMatrix are currently rejected together. Done means a Lighthouse recommended preset can provide the defaults while page-specific assertMatrix entries override selected checks without the existing error.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
performance, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.