sitespeedio / sitespeedio/browsertime
Native support for external ad/tracker blocklists (URLs/Files) to measure third-party impact
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 652
- Forks
- 141
- Avg merge
- 3h 56m
- Merged PRs (30d)
- 5
Description
Feature/improvement
Is your feature request related to a problem? Please describe.
Currently, if we want to measure the performance impact of third-party ads and trackers on a website, we have to manually specify domains using --browsertime.block or --browsertime.chrome.blockDomainsExcept.
While this works for a handful of domains, it becomes difficult to manage when trying to use large, community-maintained blocklists (like those used by Pi-hole, uBlock Origin, or AdGuard) which contain tens of thousands of domains. Passing thousands of domains manually via CLI arguments is impractical and can hit maximum command-line length limits in some environments.
while Chrome's SPOF feature (--host-resolver-rules) or a custom wrapper script can technically achieve this, there is no built-in, frictionless way to automate this within the tool itself.
Describe the solution you'd like
I propose adding native support for fetching and parsing external blocklists directly within Browsertime/sitespeed.io.
Ideally, this would introduce two new CLI flags:
1. Pass blocklist URLs directly:
--browsertime.blockListUrl https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
--browsertime.blockListUrl https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts.txt
2. Pass a file containing a list of URLs:
--browsertime.blockListFile blocklists.txt
(Where blocklists.txt contains a plain-text list of raw URLs to fetch).
Expected Behavior:
- Sitespeed fetches the provided URLs before the test starts.
- It parses the content (supporting standard
hostsfile formats or basic line-by-line domain lists, stripping comments#). - It converts them internally into the equivalent of
--browsertime.blockrules. - The browser drops these requests at the network level during the test.
Describe alternatives you've considered
- Custom Wrapper Scripts: Writing a custom Node/Bash script that downloads the lists, parses them, formats them into a massive string of
--browsertime.block domain1 --browsertime.block domain2, and executes the Docker container. (Drawback: Highly prone to hitting CLI character limits and requires everyone to reinvent the wheel). - SPOF Testing via Chrome args: Using
--chrome.args='--host-resolver-rules=...'to map trackers tounreachable.sitespeed.io. (Drawback: Browser specific, tedious to format for 50,000+ domains, and slightly different behavior than outright blocking the request).
Additional context
The primary use-case for this is Automated A/B Performance Testing.
By easily toggling these blocklists on and off, developers and performance engineers can easily quantify exactly how much third-party scripts are hurting their site's UX.
It would allow us to easily generate reports comparing:
- Baseline Page Load
- Page Load with trackers blocked
And cleanly observe the delta in metrics like:
SpeedIndex/LCPrequests.totalthirdParty.requeststransferSize.thirdPartycpu.longTasks
Bringing Pi-hole/uBlock style blocking natively to sitespeed.io would be an very useful asset for the web performance community.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how --browsertime.block and --browsertime.chrome.blockDomainsExcept are parsed and applied during browser setup. Define the URL and file input flow, fetching and parsing of hosts or domain-list formats, conversion to blocking rules, and tests showing requests are blocked before measurement begins.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- cli, performance, web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100