[App Router] AI search crawlers and TelegramBot miss streaming metadata in Edge SSR
Open
Nobody has claimed this yet.
Runtime
- Dominant language
- JavaScript
- Stars
- 142k
- Forks
- 32.4k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 351
Description
Link to the code that reproduces this issue
https://github.com/agentHits/next.js/tree/fix/seo-monolithic-head-metadata
To Reproduce
- Deploy or run an App Router page with asynchronous metadata streaming in Edge SSR runtime.
- Send an HTTP request with User-Agent
TelegramBot (like TwitterBot),ChatGPT-User,GPTBot, orClaudeBot. - Read the initial HTML chunk received before the full response streams.
Current vs. Expected behavior
- Current: Metadata tags (
<title>,<meta name="description">, OpenGraph tags) stream in subsequent Suspense chunks. Because HTML-limited bots close the socket after the first chunk, social cards in Telegram and indexing in AI crawlers show empty/missing metadata. - Expected: Next.js should recognize these user agents and emit a synchronous, monolithic
<head>with all metadata prior to the closing</head>tag.
Provide environment information
Operating System:
Platform: darwin
Arch: arm64
Version: Darwin Kernel Version 24.2.0
Binaries:
Node: 24.21.0
pnpm: 10.33.0
Relevant Packages:
next: 16.4.0-canary.32
react: 19.3.0
react-dom: 19.3.0
Which area(s) are affected? (Select all that apply)
App Router, Edge Runtime (SSR)
Which stage(s) are affected? (Select all that apply)
next start (production mode)
Additional context
Fix verified with failing/passing test in test/e2e/app-dir/metadata-streaming/metadata-streaming.test.ts and test/unit/is-bot.test.ts.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with test/e2e/app-dir/metadata-streaming/metadata-streaming.test.ts and test/unit/is-bot.test.ts, then run the reproducer against Edge SSR with the listed user agents. Trace how the initial HTML chunk and bot detection are handled. Done means the affected agents receive a synchronous monolithic head containing the metadata, with the tests passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, next.js, react
- Domain
- backend, testing-qa, web-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100