HarperFast / HarperFast/prerender-plugin

Measure k: the per-page script-driven origin calls the offload figure counts on neither side

Đang mở
#153 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
JavaScript
Star
0
Fork
0
Merge trung bình
9 giờ 39 phút
Pull request đã merge (30 ngày)
39

Mô tả

Follow-up to #152. The console states net origin offload **documents-only** and says the fifth term is
missing from **both** sides of the ledger: the XHR/API calls a page's own scripts make when a rendering
crawler runs it. This issue measures that term so the console can count it instead of stating it.

The caveat is still in the tree (`packages/console/README.md`): *"None of it passes through the plugin,
so the figure is documents-only on both sides … the true net offload for rendering crawlers is higher
than shown."*

## Status — updated 2026-09-16

**Not shipped.** #154 implemented all three stages and was **closed unmerged**; nothing from it is on
`main` (no `hydration_calls`, no `uncacheableSubrequests`, no `scriptsStripped`, no `rendersJs`). The
work below stands as designed; only the version plan needed renumbering, because the tags it reserved
were consumed by the #155–#159 release train.

| stage | was | now |
|---|---|---|
| browser | v1.22.0 | **v1.25.0** (`main` is 1.24.0) |
| plugin | v0.65.0 | **v0.68.0** (0.66.1 shipped #106; 0.67.0 is taken by #102's PR #162) |
| console | v0.13.0 | **v0.14.0** (`main` is 0.13.0 — went to #159) |

One interaction to fold in before re-implementing: #158's endpoint switch drops the page's own
price/availability XHR from bot renders via a fleet `block.urlPatterns` entry. That shrinks the
renderer's own `k` on that deployment and raises the `blocked` count — which the design already
reports, and which is exactly why `blocked` is counted separately rather than silently lost.

## The quantity

`k(page)` = same-origin subrequests a page load makes whose response **no shared cache would serve** —
the calls that reach the origin whoever runs the page. Then, per page-view by a crawler that executes
scripts:

| | origin requests |
|---|---|
| without prerender | `1 + k` (document, then the page's calls) |
| cache-served, snapshot **without** scripts | `0` — `k` is *saved* |
| cache-served, snapshot **with** scripts | `k` — *incurred* |
| proxied origin page | `1 + k` — *incurred* |
| our own render | `1 + k` — *incurred* (the renderer runs the page too) |

## Where each piece is measured

**Browser (`v1.25.0`)** — the renderer's response hook already sees every same-origin response and
already inspects cache headers for its own resource cache. Add a pure classifier over
`(request, response)` → `uncacheable` (explicit: non-GET, `no-store`/`private`/`no-cache`,
`Set-Cookie`, uncacheable status), `cacheable` (explicit positive freshness: `s-maxage`/`max-age`/
`Expires`), or `unspecified` (no freshness info — CDN-default dependent; reported, counted on neither
side). Also count same-origin requests our block list aborted (`blocked`), so the undercount is
visible. Carry the counts on the attempt and post them with the result as
`subrequests: { sameOrigin, cacheable, uncacheable, unspecified, blocked }`, plus
`scriptsStripped: ` so the plugin knows whether the stored snapshot can
hydrate at all. Cost: string checks per response; ~5 ints on the wire.

**Plugin (`prerender-v0.68.0`)**
- `PrerenderedPage` gains `uncacheableSubrequests: Int` and `scriptsStripped: Boolean` (nullable; older
rows read as unknown). Written on store.
- `render` gains series `subrequests` (method = kind) — a value per posted result, so Σ = mean × count;
this is the render fleet's own `k` cost.
- Registry: `analytics.bots[].rendersJs` (boolean). Defaults `true` for the documented renderers only —
Googlebot, Google InspectionTool, Bingbot, Applebot, YandexBot. Every AI crawler in the registry runs
nothing and stays unflagged.
- Serve path: for a request from a flagged crawler, emit `hydration_calls` (new metric; it needs its own
three slots): path = side (`saved` | `incurred` | `unknown`), method = bot, type = source, value = `k`.
`saved` = cache serve of a script-stripped snapshot; `incurred` = cache serve of a snapshot with
scripts, or any origin serve; `unknown` = no page record, or a page that predates the upgrade. One
in-memory counter bump; ~30 combo rows/period/node.

**Console (`prerender-console-v0.14.0`)** — `originLoad()` becomes
`net = 1 − (proxied + renders + Σk_renders + probes + sitemaps + Σk_incurred) ÷ (arrived + Σk_saved + Σk_incurred)`,
the exposure tile becomes "script calls: saved X · incurred Y", the `unknown` count is printed (it decays
over one render cycle after the plugin deploys), and `unspecified` is shown as the CDN-default caveat.

## Order

Every stage tolerates the others being absent (the console falls back to v0.12.0's exposure count; the
plugin ignores fields an older fleet does not post). Browser first so `k` starts landing on pages;
plugin; console. Plugin + console land in one PR here (the console's catalog guard scans the plugin's
emit sites, so a plugin-only merge would fail the console suite), released as two tags.

## Not resource-intensive

Measurement rides on hooks that already run per response; storage is two small nullable fields per page;
the serve path adds one counter bump for rendering-crawler serves. The cost is coordination and the
one-render-cycle warm-up.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start with the caveat in packages/console/README.md, then trace the browser response hook, postProcess.stripScripts, the plugin serve path, and console originLoad() described here. Done means the browser, plugin, and console stages tolerate missing peers, report saved/incurred/unknown and unspecified values, and the console suite passes with the plugin catalog guard.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
javascript
Lĩnh vực
analytics, full-stack
Loại issue
Tính năng
Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức độ hoạt động
Sôi nổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
48/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.