REST API returns HTTP 404 with a valid body when running on Next.js 16

Link to reproduction

(I can prepare a minimum reproducible example if requested — let me know.)

Describe the Bug

When running Payload v3 on Next.js 16, all REST API routes handled by the auto-generated src/app/(payload)/api/[...slug]/route.ts (e.g. /api/users/, /api/media/file/<filename>) return HTTP 404 even when Payload itself processes the request successfully and returns a valid response body.

Key characteristics:

• The response body is correct (e.g. /api/users/ returns the proper {"docs": [...], "totalDocs": ..., ...} JSON, /api/media/file/foo.webp returns the full image bytes with the right content-length)
• Only the HTTP status is wrong — it is 404 instead of 200
• Response headers include X-Powered-By: Next.js, Payload, confirming Payload is in the chain
• Response headers also include vary: rsc, next-router-state-tree, next-router-prefetch, next-router-segment-prefetch, which is unusual for an API Route Handler
• Custom Route Handlers defined directly under src/app/api/<name>/route.ts (independent of the Payload catch-all) return 200 correctly
• The issue reproduces on both production build (next build && next start with output: 'standalone') and development (pnpm dev with turbopack)
• Behavior is independent of the request Host header in the staging/dev environment; in production we observed that requests made via internal Docker network with Host: <container-name>:3000 returned 200, while any external hostname (cocorograph.co, dev.cocorograph.co, even example.com) returned 404

Practical impact:

• Front-end image rendering through /_next/image still works (Next.js's optimizer fetches the upstream, gets the bytes, and serves them anyway), so the issue is not visible in the browser
• However, any script that calls the REST API directly (e.g. curl -sI, OG image fetchers, indexing crawlers, internal automation that hits /api/<collection>/file/<filename>) sees a 404 and behaves as if the resource is missing — this caused a misdiagnosis in our content workflow

Reproduction Steps

1. Set up a Payload v3.84.1 project with the default npx create-payload-app template
2. Use next@^16.2.6, react@^19.2.6 in package.json
3. Set trailingSlash: true and output: 'standalone' in next.config.mjs
4. Create a Media collection with default upload config (local storage, staticDir: 'media'), upload a file via the admin panel
5. Build and start in production mode (or run pnpm dev)
6. curl -sI https://your-site/api/users/ → returns 404 even though body is the correct user list JSON
7. curl -sI https://your-site/api/media/file/<uploaded-filename> → returns 404 even though body is the full image bytes

Which area(s) are affected? (Select all that apply)

area: core, area: ui, area: next-payload

Environment Info

Payload:
  payload                            : 3.84.1
  @payloadcms/db-postgres            : 3.84.1
  @payloadcms/email-nodemailer       : 3.84.1
  @payloadcms/live-preview-react     : 3.84.1
  @payloadcms/next                   : 3.84.1
  @payloadcms/richtext-lexical       : 3.84.1
  @payloadcms/translations           : 3.84.1
  @payloadcms/ui                     : 3.84.1

Framework:
  next                               : 16.2.6
  react                              : 19.2.6
  react-dom                          : 19.2.6

Runtime:
  Node.js                            : 24.16.0
  Operating System                   : Ubuntu 22.04 LTS (Docker container, node:24-bookworm-slim)

Database:
  PostgreSQL                         : 18 (via @payloadcms/db-postgres)

Additional context

What we have tried (none of which fixed it):

1. Upgrading from Payload 3.81 → 3.84.1 and Next.js 16.2.1 → 16.2.6 — confirmed both old and new versions exhibit the same behavior
2. Bypassing our custom proxy.ts (Next.js 16 middleware) for /api/ paths by setting config.matcher: ['/((?!favicon\\.ico|api/).*)'] — the proxy-set headers disappear from the response (so the matcher works), but the 404 status and the vary: rsc, ... headers remain
3. Hitting the container directly via the Docker network (docker run --rm --network <net> curlimages/curl -sI http://app-container:3000/api/media/file/foo.webp) — returns 200 with Host: app-container:3000, but the same container returns 404 when any other Host header is sent
4. Confirmed .next/server/app/(payload)/api/[...slug]/route.js exists in the production build output

Hypotheses (unconfirmed):

• The vary: rsc, next-router-state-tree, ... headers suggest Next.js 16 is routing the API response through the RSC processing pipeline, which may be overwriting the status code
• The host-dependent behavior in production hints at some sort of internal cache key or self-host detection differing between in-cluster and external requests

Happy to provide a minimum reproducible repo or run further diagnostics if helpful. Thanks for the great work on Payload v3!

Start with the generated src/app/(payload)/api/[...slug]/route.ts and compare it with a custom src/app/api//route.ts under Next.js 16. Review next.config.mjs and proxy.ts, then reproduce with the listed curl requests in development and production builds, varying the Host header. Done means Payload REST routes return HTTP 200 with their existing valid bodies and expected headers.