HarperFast / HarperFast/prerender-plugin

Resource cache stores only third-party tag/personalization scripts (0 first-party) — it should be restricted by host

Open
#108 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
0
Forks
0
Avg merge
9h 39m
Merged PRs (30d)
39

Description

## What happens

`ResourceCache` will store any host's script or stylesheet that passes the HTTP policy check, so
third-party tag-manager, A/B-testing and personalization scripts get cached alongside the site's own
assets. There is no host restriction of any kind.

Measured 2026-09-16 against a live storefront, with the cache warm: a mobile home-page render stores
**54 entries / 5.1 MB, and every one of them is third-party — not a single first-party asset.** Across
home + a product page it is 69 entries, same story:

| host | entries |
|---|---:|
| `assets.adobedtm.com` (Adobe Launch + its rule bundles) | 47 |
| `display.ugc.bazaarvoice.com` / `apps.` / `analytics-static.` | 6 |
| `cdn.cookielaw.org` (OneTrust) | 2 |
| `pub.loudcrowd.com` | 2 |
| one each: personalization vendor, Clicktale, Constructor, Taboola, PubMatic, ID5, GTM, impactradius, coherentpath, cnnx, fohr, Google shopping agent | 12 |
| **first-party (`*.`)** | **0** |

`CACHEABLE_RESOURCE_TYPES` is `['stylesheet','script']`, so the stored population is bundles — not the
XHR/fetch responses a vendor delivers its per-session decisions over.

#109 is *why* there are no first-party entries, and the two compound: that issue excludes everything
safe to cache, this one keeps everything that isn't.

## Mechanism

`isCacheableRequest` (`packages/browser/src/ResourceCache.ts`) gates on method, navigation, resource
type and auth/cookie headers only:

```js
isCacheableRequest(req) {
if (req.method() !== 'GET') return false;
if (req.isNavigationRequest()) return false;
if (!CACHEABLE_RESOURCE_TYPES.has(req.resourceType())) return false;
const headers = req.headers();
if (headers['authorization'] || headers['cookie']) return false;
return true;
}
```

Anything with a positive `max-age` is then eligible, and tag vendors serve their bundles with long
max-ages (observed: 1200s to 30 days).

## The content loss this issue was filed for has been explained, and fixed

**Filed on the hypothesis that replaying session-scoped scripts changes the snapshot.** The measurement
was real — on a live storefront's mobile home page, with the cache enabled, the served HTML lost
~5,290 characters of text, 22 links and 16 images on every render, interleaved off/on/off/on with the
arms internally identical. The *attribution* was wrong.

`c515d76` (browser **v1.24.0**) found the actual mechanism: a cached response whose headers **repeat**
reaches us as puppeteer's `\n`-join of the values, and CDP refuses to fulfil a response carrying one
(`Fetch.fulfillRequest` → `Invalid header: `, rejecting the *whole* payload). The rejection was
swallowed (`.catch(noop)`), so the request was never answered at all — the resource silently never
loaded, for the rest of the render. Identical signature: content missing, timing unchanged, 200,
non-empty, nothing logged.

**Confirmed by reverting the fix** (2026-09-16, same page, same interleave, released v1.24.0 build with
both halves of `c515d76` reverted to their 1.16.0 behaviour):

| build | text | links | imgs | bytes | `cacheReplaysRefused` |
|---|---|---|---|---|---|
| v1.24.0 as released | 20,641 → 20,641 (0) | 421 → 421 (0) | 250 → 250 (0) | +1 | **0** |
| fix reverted | 20,641 → **18,800** | 421 → **418** | 250 → **247** | **−48,864** | **8** (2/render) |

Arms internally identical (spread 0), so every delta is real by this issue's own bar. Three of the 54
cached entries carry a repeated header and two are refused per render:

```
d.impactradius-event.com/…/A375953-….js x-goog-hash (2 values)
agents.cloud.google.com/shopping_agent/static.js content-security-policy (3 values)
cdn.cookielaw.org/consent/…/otSDKStub.js access-control-expose-headers (2 values)
cross-origin-resource-policy (2 values)
```

The −48,864 bytes independently reproduces the "~49 KB smaller" `c515d76` measured on a product page.

## What is still open

**The host restriction.** It no longer rests on a demonstrated content loss; it rests on this: the
cache's entire value is the site's own bundles (megabytes per render), third-party tags contribute
**nothing** to the snapshot, and on this deployment they are **100%** of what it stores. Three of them
were actively costing content until this week. Caching them is all risk and no return.

Proposal unchanged: `resourceCache.hosts: string[]`, matched as host-or-subdomain suffixes, empty list
preserving today's cache-any-host behaviour. The library can't infer the right set — a storefront's
asset host is frequently a different registrable domain from the navigation host — so a same-site
default derived from the navigation URL handles the simple case but can't be the only mechanism.

Worth deciding whether the restrictive behaviour should be the default. A deployment that already
blocks tag hosts through `block.urlPatterns` never sees any of this, which is a good way for it to stay
unnoticed.

**Untested, and no longer load-bearing for this issue: whether a stale configuration bundle changes the
snapshot.** Only script/stylesheet are cached, so a per-session decision fetched over XHR cannot be
replayed; what *can* go stale is a rule bundle that bakes decisions in — which is precisely what those
47 `assets.adobedtm.com/.../RC*.js` entries are. Testing it needs an **aged** cache (warm on a pod,
replay a day later) or a diff of a replayed bundle against a fresh fetch. A cache warmed minutes before
the replay cannot see it, and will read as a clean null result.

## How to test for it

Interleave the arms (off/on/off/on) rather than running all-off then all-on: these pages vary on their
own between renders, and an A,A,B,B ordering attributes site-side session variance to the cache. Then
treat a cross-arm difference as real only when it exceeds the within-arm spread. Comparing text length,
anchor count, image count and JSON-LD count of the served HTML is enough; a raw byte diff is too noisy.
Read `cacheReplaysRefused` (added in v1.24.0) alongside — a non-zero count means resources are being
dropped rather than replayed.

Contributor guide

Open the contributing guide

Research direction

Start with packages/browser/src/ResourceCache.ts and its isCacheableRequest entry point; review how resourceCache configuration is defined and how cache hosts are currently handled. Define completion around host-or-subdomain matching, empty-list compatibility, and tests covering the chosen default behavior and replay eligibility.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.