cloudflare / cloudflare/vinext

Dev cache parity: make `cacheMaxMemorySize: 0` and custom cache handlers fast in dev (TieredCacheHandler)

Open
#2,110 0 comments 0 reactions 0 assignees View on GitHub
nextjs-tracking
Dominant language
TypeScript
Stars
8.8k
Forks
406
Avg merge
2d 6h
Merged PRs (30d)
120

Description

## Summary

Next.js changed dev-mode cache behavior so that `cacheMaxMemorySize: 0` and custom cache handlers no longer cause warm reloads to be treated as cold cache misses under Cache Components. The fix introduces a built-in in-memory front handler that fronts the configured (slow/remote) backing handler in dev only, so warm reads resolve in a microtask.

Before the fix, when Cache Components is enabled, `next dev` treated a `'use cache'` value as a miss whenever the read didn't resolve right away. Two dev configurations triggered this even for cached entries:

1. `cacheMaxMemorySize: 0` replaced the built-in default handler with a no-op stub
2. Custom cache handlers with a slow or remote `get` didn't return in time

## Behavior

- **Size-0 case:** Dev now uses a real in-memory handler instead of the no-op stub. The `'use cache'` wrapper forces a dynamic cache life for it (`revalidate: 0`, 5-minute `expire`) — same treatment private caches receive — so every read serves the stale entry and re-warms in the background. Also fixes the dev private handler, which was sized from `cacheMaxMemorySize` and degraded to the no-op stub when set to 0.
- **Custom handler case:** A new `TieredCacheHandler` puts a fast built-in in-memory front in front of the user-configured backing. Warm reads serve from the front, writes go through to both tiers, and the front reconciles against the backing in the background (evicting front entries when the backing no longer has them, via an already-expired overwrite since the handler interface has no per-key delete).
- Dev-only handlers are gated on `process.env.__NEXT_DEV_SERVER`; production is unchanged (`cacheMaxMemorySize: 0` still caches nothing, configured handlers used directly).
- Dev-only handlers are kept out of the main registered handler set but merged in where tag operations iterate, so `revalidateTag` still reaches them.

## vinext implications

vinext's dev cache layer (see `isr-cache.ts` / `cache-handler` config) should consider:

- Whether vinext exhibits the same dev cold-miss behavior under the new `'use cache'` semantics when `cacheMaxMemorySize: 0` is configured or when a slow custom handler is used
- Whether to implement an equivalent dev-only front handler / tiered handler so warm reads in vinext dev don't surface as cold-cache misses
- Tag invalidation must continue to reach the dev front handler if we add one

This is dev-only parity but matters for users who configure `cacheMaxMemorySize: 0` (e.g. to emulate deploy envs) or who plug in a remote cache handler.

## Upstream

- Commit: vercel/next.js@96e9a8e246a7a9599793226d6fbcdb0db3655f6e
- PR: https://github.com/vercel/next.js/pull/94784
- Key files: `packages/next/src/server/use-cache/tiered-cache-handler.ts` (new), `packages/next/src/server/use-cache/handlers.ts`, `packages/next/src/server/use-cache/use-cache-wrapper.ts`
- Tests: `test/development/app-dir/use-cache-size-zero/`, `test/development/app-dir/use-cache-custom-handler-dev/`

Related: #1936 (dev cache serve-until-expire), #1937 (persist `use cache: private` in dev)

Contributor guide

Open the contributing guide

Research direction

Start by reading vinext's isr-cache.ts and cache-handler configuration, then compare them with the upstream handlers.ts, tiered-cache-handler.ts, and use-cache-wrapper.ts. Run the use-cache-size-zero and use-cache-custom-handler-dev test directories to establish current behavior. Done means warm dev reads avoid cold-cache misses for both configurations while tag invalidation still reaches the relevant handlers, without changing production behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
next.js, typescript
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.