sitespeedio / sitespeedio/browsertime

Native support for external ad/tracker blocklists (URLs/Files) to measure third-party impact

Open
#2,364 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
652
Forks
141
Avg merge
3h 56m
Merged PRs (30d)
5

Description

Feature/improvement
Is your feature request related to a problem? Please describe.

Currently, if we want to measure the performance impact of third-party ads and trackers on a website, we have to manually specify domains using --browsertime.block or --browsertime.chrome.blockDomainsExcept.

While this works for a handful of domains, it becomes difficult to manage when trying to use large, community-maintained blocklists (like those used by Pi-hole, uBlock Origin, or AdGuard) which contain tens of thousands of domains. Passing thousands of domains manually via CLI arguments is impractical and can hit maximum command-line length limits in some environments.

while Chrome's SPOF feature (--host-resolver-rules) or a custom wrapper script can technically achieve this, there is no built-in, frictionless way to automate this within the tool itself.

Describe the solution you'd like

I propose adding native support for fetching and parsing external blocklists directly within Browsertime/sitespeed.io.

Ideally, this would introduce two new CLI flags:

1. Pass blocklist URLs directly:

--browsertime.blockListUrl https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
--browsertime.blockListUrl https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts.txt

2. Pass a file containing a list of URLs:

--browsertime.blockListFile blocklists.txt

(Where blocklists.txt contains a plain-text list of raw URLs to fetch).

Expected Behavior:

  1. Sitespeed fetches the provided URLs before the test starts.
  2. It parses the content (supporting standard hosts file formats or basic line-by-line domain lists, stripping comments #).
  3. It converts them internally into the equivalent of --browsertime.block rules.
  4. The browser drops these requests at the network level during the test.
Describe alternatives you've considered
  1. Custom Wrapper Scripts: Writing a custom Node/Bash script that downloads the lists, parses them, formats them into a massive string of --browsertime.block domain1 --browsertime.block domain2, and executes the Docker container. (Drawback: Highly prone to hitting CLI character limits and requires everyone to reinvent the wheel).
  2. SPOF Testing via Chrome args: Using --chrome.args='--host-resolver-rules=...' to map trackers to unreachable.sitespeed.io. (Drawback: Browser specific, tedious to format for 50,000+ domains, and slightly different behavior than outright blocking the request).
Additional context

The primary use-case for this is Automated A/B Performance Testing.
By easily toggling these blocklists on and off, developers and performance engineers can easily quantify exactly how much third-party scripts are hurting their site's UX.

It would allow us to easily generate reports comparing:

  • Baseline Page Load
  • Page Load with trackers blocked

And cleanly observe the delta in metrics like:

  • SpeedIndex / LCP
  • requests.total
  • thirdParty.requests
  • transferSize.thirdParty
  • cpu.longTasks

Bringing Pi-hole/uBlock style blocking natively to sitespeed.io would be an very useful asset for the web performance community.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing how --browsertime.block and --browsertime.chrome.blockDomainsExcept are parsed and applied during browser setup. Define the URL and file input flow, fetching and parsing of hosts or domain-list formats, conversion to blocking rules, and tests showing requests are blocked before measurement begins.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
cli, performance, web-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.