Next 16.3 regression: htmlLimitedBots matching brows... | vercel/next.js

Nobody has claimed this yet.

Link to the code that reproduces this issue

https://github.com/mdotk/next-163-html-limited-cache-repro

To Reproduce

Clone the public reproduction repository.
Run npm install.
Run npm run build.
Run npm start.
In another terminal, run npm run check.
Observe the two requests made without a user agent, with a Chrome user agent, and with a Googlebot user agent.
Change only the next dependency from 16.3.0 to 16.2.11, reinstall, rebuild, and repeat the check.

The reproduction uses:

const nextConfig = {
  cacheComponents: true,
  htmlLimitedBots: /.*/,
}

It contains a dynamic route with a partial generateStaticParams result, cached route data using "use cache" and cacheLife("minutes"), and cached asynchronous metadata.

Current vs. Expected behavior

The on-demand PPR route behaves as follows:

Request	Next 16.2.11	Next 16.3.0	Next 16.3.1-canary.0
No `User-Agent`	`s-maxage=60, stale-while-revalidate=3540`; cache HIT	`s-maxage=60, stale-while-revalidate=3540`; cache HIT	`s-maxage=60, stale-while-revalidate=3540`; cache HIT
Chrome	`s-maxage=60, stale-while-revalidate=3540`; cache HIT	`private, no-cache, no-store, max-age=0, must-revalidate`	`private, no-cache, no-store, max-age=0, must-revalidate`
Googlebot	private/no-store	private/no-store	private/no-store

All requests return HTTP 200 with the route-specific title in the initial <head>. No metadata-boundary or hydration error is emitted on 16.3.

The current behavior makes every non-empty user agent matched by the custom pattern force a dynamic/non-SSG response. Ordinary browser traffic therefore loses the route's public cache policy.

Expected behavior:

An otherwise cacheable PPR route should retain its public cache policy for ordinary browser requests.
Matching HTML-limited agents should receive complete critical metadata in the initial head.
The prerender and resume metadata trees should remain compatible.
No metadata-boundary or client-render fallback should occur.

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin 25.5.0
  Available memory (MB): 24576
  Available CPU cores: 12
Binaries:
  Node: 20.20.2
  npm: 10.9.2
  Yarn: N/A
  pnpm: N/A
Relevant Packages:
  next: 16.3.0
  eslint-config-next: N/A
  react: 19.2.3
  react-dom: 19.2.3
  typescript: N/A
Next.js Config:
  cacheComponents: true
  htmlLimitedBots: /.*/

The result was also reproduced on next@16.3.1-canary.0.

Which area(s) are affected? (Select all that apply)

cacheComponents
Dynamic Routes
Metadata
Partial Prerendering (PPR)
Performance
Runtime
Use Cache

Which stage(s) are affected? (Select all that apply)

next build (local)
next start (local)

Additional context

The behavior appears to come from the combined changes in #96364 and #96367.

The 16.3 App Page runtime effectively selects:

const serveStreamingMetadata =
  !userAgent ? true : shouldServeStreamingMetadata(...)

const shouldForceDynamicPPRRender =
  isRoutePPREnabled && !serveStreamingMetadata

const isSSG =
  (prerenderManifestEntry || staticPath) &&
  !shouldForceDynamicPPRRender

The build also converts the configured htmlLimitedBots regex into a user-agent experimentalBypassFor rule.

For htmlLimitedBots: /.*/:

Every non-empty user agent disables streaming metadata.
Every matched PPR request sets shouldForceDynamicPPRRender.
The request is no longer treated as SSG/prerender-cache eligible.
The response reaches the revalidate: 0 path.
getCacheControlHeader({ revalidate: 0 }) emits private, no-cache, no-store, max-age=0, must-revalidate.

This explains why no-UA remains cacheable while every non-empty user agent does not.

The change in #96364 prevents a blocking-metadata request from resuming a prerender tree created with the incompatible streaming-metadata shape. Simply restoring the previous isSSG behavior could therefore reintroduce the metadata-tree mismatch.

A durable fix likely needs to separate blocking-metadata selection from unconditional prerender-cache bypass, gate the bypass on whether route metadata is actually postponed or incompatible with the cached shell, or expose independent controls for these decisions.

Related work:

#93401 — original custom htmlLimitedBots PPR metadata mismatch
#94630 — resume metadata-tree compatibility approach
#95406 — custom-only bot mismatch and missing metadata on 16.2
#96364 — force dynamic PPR render for configured HTML bots
#96367 — configured htmlLimitedBots cache-components bypass
#96584 — substring semantics for generated bypass rules; does not change the /.*/ case
#96593 — previous report automatically closed because it lacked the now-published reproduction link

On a CDN configured to respect origin cache headers, this behavior prevents ordinary browser responses from converging to edge cache HITs. On high-cardinality public sites that can materially increase origin traffic, rendering concurrency, and memory pressure.

Clone the linked reproduction, run its npm install/build/start/check steps, and compare Next 16.3.0 with 16.2.11 using the reported User-Agent requests. Then inspect the changes referenced in #96364 and #96367 and the related metadata issues. Done means ordinary browser requests retain public cache headers while HTML-limited agents receive complete initial metadata without metadata-boundary or hydration failures.

Written by the indexing model from the issue text.

Next 16.3 regression: htmlLimitedBots matching browser UAs makes cacheable PPR pages private/no-store

Description

Link to the code that reproduces this issue

To Reproduce

Current vs. Expected behavior

Provide environment information

Which area(s) are affected? (Select all that apply)

Which stage(s) are affected? (Select all that apply)

Additional context

Contributor guide

First steps

Research direction

Assessment

Next 16.3 regression: htmlLimitedBots matching browser UAs makes cacheable PPR pages private/no-store

Description

Link to the code that reproduces this issue

To Reproduce

Current vs. Expected behavior

Provide environment information

Which area(s) are affected? (Select all that apply)

Which stage(s) are affected? (Select all that apply)

Additional context

Contributor guide

First steps

Research direction

Assessment

Get new issues in your inbox