cloudflare / cloudflare/vinext

Cache: reuse completed 'use cache' entries for the rest of a request (fix double execution of private caches on sequential reads)

Open
#2,820 0 comments 0 reactions 0 assignees View on GitHub
nextjs-tracking
Dominant language
TypeScript
Stars
8.8k
Forks
406
Avg merge
2d 6h
Merged PRs (30d)
120

Description

## Next.js Change

**Commit:** [`1ff6a80`](https://github.com/vercel/next.js/commit/1ff6a80358939e8808926aecf4b7b95c31fce355)
**PR:** [#96727](https://github.com/vercel/next.js/pull/96727)

## What changed

Fixes duplicated cache-function execution within a single request. Calling the same `'use cache: private'` function twice in one request executed its body twice in production. Preloading at the top of a segment and reading the same function again lower down for composability — the shape that motivates a preload in the first place — therefore did the work twice instead of once.

The intra-request dedupe map dropped an entry as soon as its fill completed, so it only ever covered *concurrent* invocations. A later (sequential) invocation fell through to the cache handler, and the `React.cache` memo wrapping every cache function missed whenever the arguments were not reference-equal. Public caches got a handler hit out of that, but **private caches have no handler in production** and their entries are excluded from the immutable Resume Data Cache of a dynamic request, so nothing had stored the entry — the body re-ran.

### Mechanism (from the diff)

- Completed invocations now move into a `completedCacheInvocations` map on the work store instead of being dropped; a later invocation joins that entry. The pending map keeps its previous semantics (a concurrent joiner shares a genuinely in-flight fill and must not re-run discard checks against it), whereas a completed entry is a stored value and is only reused when the caller has not asked to bypass caches and nothing has invalidated it since.
- Private caches still get no cache handler in production, so this map is what backs them. It lives on the work store, so it cannot carry request-derived data beyond the request that produced it.
- **Retention is limited to kinds where it saves real work**: private caches, and kinds whose handler came from the platform or from `cacheHandlers` config (where a read can be a network round trip). A new `isBuiltInCacheHandler(kind)` in `use-cache/handlers.ts` answers this by tracking the actual handler **instances** that `initializeCacheHandlers` constructs (via a new `@next/cache-handlers-built-in` `Set`), rather than tracking kinds.
- Tracking instances keeps aliasing correct: a self-hosted `remote` resolves to the very same in-memory handler as `default`, so its reads are cheap map lookups and its entries are *not* retained (retaining would duplicate that cache for nothing); a platform-supplied `remote` is a distinct instance and *is* retained.

## Impact on vinext

vinext reimplements `'use cache'` (including `'use cache: private'`) and cache-handler backends. If vinext's intra-request dedupe only covers concurrent calls (dropping entries on completion), sequential re-reads of the same private cache function in one request will re-execute the body — especially since private caches have no persistent handler in production. This defeats preload-then-read composability and doubles work.

What to check/do:

1. **Retain completed cache invocations for the rest of the request**, not just concurrent ones. Move completed fills into a request-scoped `completedCacheInvocations`-style map so a later sequential read joins the stored value.
2. **Keep pending vs completed semantics distinct.** A concurrent joiner shares an in-flight fill (skip discard checks); a completed entry is a stored value and should only be reused when the caller isn't bypassing caches and nothing has invalidated it since.
3. **Back private caches with this map in production** (they have no handler), scoped to the work store so nothing leaks across requests.
4. **Only retain where it saves work.** Retain private caches and entries from platform/config-supplied handlers (network reads). Skip retaining entries from the built-in in-memory handler, whose reads are cheap lookups.
5. **Track handler instances, not kinds**, to keep aliasing correct (e.g. self-hosted `remote` == `default`).
6. **Port the test.** Next.js's `use-cache-custom-handler` suite counts `::get` calls to assert two sequential reads reach the handler once; new `dedup-sequential` and `private-dedup-sequential` fixtures pin the behavior.

## Related

- #2819 — Cache: discard only entries that predate a tag revalidation
- #1937 — Persist `'use cache: private'` entries in dev
- #1936 — Dev cache handler: serve stale `'use cache'` entries until `expire`
- #2110 — Dev cache parity: make `cacheMaxMemorySize: 0` and custom handlers fast in dev

Contributor guide

Open the contributing guide

Research direction

Start by locating vinext's use-cache implementation, request work store, and cache-handler initialization, then compare their pending and completed invocation behavior with Next.js's use-cache-custom-handler suite. Port the dedup-sequential and private-dedup-sequential cases, including handler-instance aliasing, and verify that sequential private reads execute once without retaining values across requests.

Written by the indexing model from the issue text.

Assessment

Tech stack
next.js, typescript
Domain
backend, performance, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.