GoogleChrome / GoogleChrome/lighthouse
add method to update test traces/artifacts with modern Chrome
- Dominant language
- JavaScript
- Stars
- 30.8k
- Forks
- 9.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 19
Description
two TODOs for this:
- https://github.com/GoogleChrome/lighthouse/blob/1b843de669e18c144422561010985d0dde05d2c6/core/lib/tracehouse/trace-processor.js#L88-L89
- https://github.com/GoogleChrome/lighthouse/pull/14287/files#diff-5a3b0489bd666c8cb3258c1713d8251a9e1f5bb7c91daa37c71e3bba0111fc2aR519-R520
Some of the [test traces/devtoolsLogs](https://github.com/GoogleChrome/lighthouse/tree/1b843de669e18c144422561010985d0dde05d2c6/core/test/fixtures/traces) are old enough now that they're interfering with code we write by forcing it to handle cases that haven't been relevant for years.
Main issues with easy trace updates:
- they are used in _many_ tests, and any replacements would need to be sure to cover approximately the same test surface area (e.g. if a trace is of a page with redirects that are tested for somewhere, any replacement would also need to have redirects)
- many of them were generated for precisely the purpose of their original PR from bespoke pages with no hint of how to replicate today
To avoid getting back into this situation in a few years, we discussed creating test pages and a method to update test traces. Similar to `yarn update:sample-artifacts` this will likely still be noisy and fiddly, but it should be much easier to update than trying to parse what was special about a trace from just the name "jumpy-cls-m90.json" and stepping through blame.
There are a lot of test traces, so we can start gradually, e.g. the first TODO above mentions including fallback code for pre-m67 test traces, which is just a handful of them.
The test pages could live in the Lighthouse repo (they could be the smoke tests, even), but we should consider hosting them in https://github.com/ChromeDevTools/performance-stories and sharing them with DevTools perf panel testing.
Contributor guide
Assessment
This issue has not been assessed yet.