GoogleChrome / GoogleChrome/lighthouse

Expand third-party classification to include unrecognized entities

Open
#14,440 4 comments 1 reaction 1 assignee Assigned to @alexnj View on GitHub
P3
Dominant language
JavaScript
Stars
30.8k
Forks
9.8k
Avg merge
1d 20h
Merged PRs (30d)
19

Description

Lighthouse Third Party Summary audit currently drops third-party origins that don't have an entity match in [third-party-web](https://github.com/patrickhulce/third-party-web) library. This causes several third-party origins that have either high transfer size, or main thread blocking (or both) to be dropped out from Third-party summary audit as a side effect. For example, if we audit `theverge.com` homepage, `blismedia.com`, `narrativ.com`, `vercel-insights.com` are not part of the third-party summary audit. Numerous (newer) TikTok CDNs also drop out despite their large transfer size, as the data-set is not updated with the recent changes the origin has made. There is a clear process defined by `third-party-web` to [update this data](https://github.com/patrickhulce/third-party-web#updating-the-entities), as a manual operation. This is a disconnected process and perhaps could be improved upon if we model it as a feedback loop.

Granted this issue is not severe when the same asset is detected elsewhere for transfer size or execution time in other audits that focus on those. Thus from an issue-detection perspective, the user might find the third-party impact in other audits. However from a proper third-party classification perspective, this could be improved.

To show an example of a third-party that does block main thread quite long, but gets excluded from third-party summary, here's a test-case:
* [http://lighthouse-thirdparty-mtb.alexnjose.com/](http://lighthouse-thirdparty-mtb.alexnjose.com/) is a first party that integrates `mtb-thirdparty.surge.sh`
* `mtb-thirdparty.surge.sh` blocks main thread for 2000+ ms.

Currently, this is what the summary audit of this test-case origin would produce (and it passes):

![image](https://user-images.githubusercontent.com/683500/194949132-b266a5c2-dfe5-4a33-8d4e-4a69a5afd6bf.png)

where it should've been the following (and fail due to high blocking time):

![image](https://user-images.githubusercontent.com/683500/194949509-f072d0a3-ac75-4145-8034-ceeb128be5a3.png)

## Proposal

I think one option we could pursue is to make up a third-party entity based on the root level domain, and not drop unrecognized third-party domains. This should be fairly straightforward by maintaining an in-memory lookup table of entities recognized during the audit, while maintaining compatibility with the `IEntity` interface exposed by third-party-web.

The drawback with this approach is the duplication of already-recognized entities with their new, unrecognized origins. TikTok is an example of a recognized entity, and a duplicate entry would be created for a CDN host that's not recognized (example today: `ttwstatic.com`, `tiktokcdn-us.com`, etc.). This could be improved further as below:

### Closing the loop with third-party-web

One option to reintegrate these unrecognized entities is to help the user contribute back to third-party-web. We could mark up the unrecognized links in the report, as below:

![image](https://user-images.githubusercontent.com/683500/194950630-8cbe133b-3127-4e30-a36c-21993416205e.png)

We could use GitHub issue-creation link to automatically fill in the required title and meta data.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.