vercel / vercel/next.js

Concurrent FileSystemCache writes can corrupt fetch-cache JSON on a shared filesystem

Open
#98,009 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Runtime
Dominant language
JavaScript
Stars
142k
Forks
32.5k
Avg merge
2d 14h
Merged PRs (30d)
351

Description

Link to the code that reproduces this issue

https://github.com/Little-Working/nextjs-shared-cache-race-repro

To Reproduce
  1. Clone the linked reproduction and run corepack enable.
  2. Run pnpm install.
  3. Run pnpm repro.
  4. The script starts two concurrent Next.js MultiFileWriter instances writing differently sized JSON values to the same cache path through a shared/network-filesystem-like adapter, while another task continuously reads and parses that path.
  5. Observe a positive parseErrors count. The final file can also be invalid JSON.

Results from single runs:

  • next@15.5.15: 1,835 parse failures.
  • next@16.4.0-canary.9: 1,613 parse failures (a repeat run produced 1,748).

The production failure that led to this reproduction occurs with multiple next start/standalone instances sharing the same .next/cache directory on an RWX network filesystem.

Current vs. Expected behavior

Current behavior

Next.js' filesystem cache writes serialized fetch-cache entries directly to their final path. In MultiFileWriter.append(), the write is scheduled as:

const promise = task[1].then(() => this.fs.writeFile(filePath, data))

When two server instances regenerate the same cache key concurrently, both can truncate and write the same destination. A reader can then observe an empty, partial, or mixed JSON document. In production this surfaces from the FileSystemCache read path as errors such as:

SyntaxError: Expected ':' after property name in JSON at position 866698
    at JSON.parse (<anonymous>)

Time-based or tag revalidation does not serialize writers across processes.

Expected behavior

A cache entry should become visible atomically: write the complete value to a uniquely named temporary file in the same directory, then rename it over the destination. Concurrent updates may remain last-writer-wins, but readers should only observe the previous complete value or the next complete value, never an in-progress write.

Next.js already contains packages/next/src/lib/fs/write-atomic.ts, which implements this temp-file-plus-rename strategy and is used by other Next.js file writes. However, the runtime FileSystemCache path goes through MultiFileWriter, which still calls fs.writeFile on the final path. Is there a reason the existing atomic writer is not used for FileSystemCache entries? If the injected filesystem abstraction is the constraint, could atomic replacement be added to that interface?

Provide environment information
Operating System:
  Platform: linux
  Arch: x64
  Version: Kubernetes containers based on the official node:22 image
Binaries:
  Node: 22.x
  pnpm: 11.1.3
Relevant Packages:
  next: 15.5.15
  next (canary verification): 16.4.0-canary.9
  react: 19.1.0 (production application)
  react-dom: 19.1.0 (production application)
  typescript: 5.8.3 (production application)
Next.js Config:
  output: standalone
Storage:
  Shared RWX network filesystem mounted as .next/cache
Which area(s) are affected? (Select all that apply)

Use Cache, Runtime

Which stage(s) are affected? (Select all that apply)

Other (Deployed)

Additional context

The production topology uses multiple Docker/Kubernetes replicas with the same writable .next/cache/fetch-cache directory on a shared filesystem. The affected value is produced by unstable_cache; the exception is raised while FileSystemCache parses the persisted entry.

This report is specifically about runtime Data Cache/FileSystemCache entries. Issues #96664 and #96259 appear related in that they also involve concurrent non-atomic JSON writes, but they concern development-time prerender manifests rather than the production fetch cache.

The deterministic reproduction uses a deliberately slow filesystem adapter to model the visibility characteristics of a network filesystem and make the race complete quickly. The same failure occurs nondeterministically with ordinary concurrent writers on shared storage.

Issue #97271 reports the same non-atomic fetch-cache write path, but it was automatically closed and locked because no valid public reproduction link was provided. This report supplies a public minimal reproduction in the template's required field, as requested by the auto-close guidance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with MultiFileWriter.append() and packages/next/src/lib/fs/write-atomic.ts, then run the linked reproduction with corepack enable, pnpm install, and pnpm repro. Trace the injected filesystem operations and verify that concurrent FileSystemCache writes leave readers seeing only complete JSON values, with no parse errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nextjs, node.js
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
67/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.