sitespeedio / sitespeedio/sitespeed.io

adding to extra's field in plugin makes comparison mode fail

Open
#4,292 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
JavaScript
Stars
5k
Forks
624
Avg merge
4h 8m
Merged PRs (30d)
2

Description

Have you read the documentation?
URL

http://example.com/

What are you trying to accomplish

Trying to add custom metrics via a plugin, the end goal is to have the comparison complete and eventually
display the custom metrics on the comparison page (if possible)

right now the problem is that the second comparison test fails with the following error:

ERROR: TypeError: currentMetric.getValues is not a function
at ComparePlugin.processMessage (file:///usr/src/app/lib/plugins/compare/index.js:174:37)

What browser did you use?

Chrome

How to reproduce
Go to the following example repository

https://github.com/dbg-daniel-aurora/simple-sitespeedio-plugin 

clone and run ./start-plugin-test.sh

Or

create a plugin that adds to the extra's field for a custom metric from within a postURLScript.

Run a test and save the baseline

Run a test with the baseline

look at output
Log output
Chromium 107.0.5304.62 Ubuntu 22.04
Mozilla Firefox 130.0.1
[2024-10-10 23:16:51] INFO: Versions OS: linux 6.10.4-linuxkit nodejs: v20.17.0 sitespeed.io: 35.1.2 browsertime: 23.1.1 coach: 8.0.2
[2024-10-10 23:16:51] INFO: Starting the compare plugin. Will save this test as the baseline
[2024-10-10 23:16:51] WARN: You should use 20+ iterations to get statistical significant data
[2024-10-10 23:16:52] INFO: Running tests using Chrome - 1 iteration(s)
[2024-10-10 23:16:52] INFO: Testing url https://www.example.com/ iteration 1
[2024-10-10 23:17:01] INFO: Take after page complete check screenshot
[2024-10-10 23:17:01] INFO: Take cumulative layout shift screenshot
[2024-10-10 23:17:01] INFO: Take largest contentful paint screenshot
[2024-10-10 23:17:04] INFO: Use the visual metrics portable script
[2024-10-10 23:17:04] INFO: Get visual metrics from the video
[2024-10-10 23:17:05] INFO: https://www.example.com/ 2 requests, TTFB: 110ms, firstPaint: 150ms, firstVisualChange: 156ms, FCP: 150ms, DOMContentLoaded: 117ms, LCP: 150ms, CLS: 0, TBT: 0ms, CPUBenchmark: 53ms, Load: 117ms, speedIndex: 156ms, visualComplete85: 156ms, lastVisualChange: 156ms
[2024-10-10 23:17:05] INFO: The server responded with a 404 status code for https://www.example.com/favicon.ico
[2024-10-10 23:17:05] INFO: Using id test_page for page baseline
[2024-10-10 23:17:06] INFO: HTML stored in /sitespeed.io/sitespeed-result/www.example.com/2024-10-10-23-16-51
Chromium 107.0.5304.62 Ubuntu 22.04
Mozilla Firefox 130.0.1
[2024-10-10 23:17:06] INFO: Versions OS: linux 6.10.4-linuxkit nodejs: v20.17.0 sitespeed.io: 35.1.2 browsertime: 23.1.1 coach: 8.0.2
[2024-10-10 23:17:06] INFO: Starting the compare plugin.
[2024-10-10 23:17:06] WARN: You should use 20+ iterations to get statistical significant data
[2024-10-10 23:17:07] INFO: Running tests using Chrome - 1 iteration(s)
[2024-10-10 23:17:07] INFO: Testing url https://www.example.com/ iteration 1
[2024-10-10 23:17:15] INFO: Take after page complete check screenshot
[2024-10-10 23:17:16] INFO: Take cumulative layout shift screenshot
[2024-10-10 23:17:16] INFO: Take largest contentful paint screenshot
[2024-10-10 23:17:19] INFO: Use the visual metrics portable script
[2024-10-10 23:17:19] INFO: Get visual metrics from the video
[2024-10-10 23:17:20] INFO: https://www.example.com/ 2 requests, TTFB: 113ms, firstPaint: 153ms, firstVisualChange: 156ms, FCP: 153ms, DOMContentLoaded: 120ms, LCP: 153ms, CLS: 0, TBT: 0ms, CPUBenchmark: 64ms, Load: 120ms, speedIndex: 156ms, visualComplete85: 156ms, lastVisualChange: 156ms
[2024-10-10 23:17:20] INFO: The server responded with a 404 status code for https://www.example.com/favicon.ico
[2024-10-10 23:17:20] INFO: Using id test_page for page baseline
[2024-10-10 23:17:20] INFO: Got a baseline:test_page-1
[2024-10-10 23:17:20] ERROR: TypeError: currentMetric.getValues is not a function
    at ComparePlugin.processMessage (file:///usr/src/app/lib/plugins/compare/index.js:174:37)
[2024-10-10 23:17:21] INFO: HTML stored in /sitespeed.io/sitespeed-result/www.example.com/2024-10-10-23-17-06

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with the linked simple-sitespeedio-plugin repository and its start-plugin-test.sh script. Read lib/plugins/compare/index.js around line 174, then run a baseline followed by a comparison with a custom metric added to extra. Done means comparison completes without the getValues error and the issue's custom-metric behavior is addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.