RSC and CDN interaction makes next.js inefficient for highload projects

Link to the code that reproduces this issue

https://codesandbox.io/p/devbox/rsc-test-m43xq4

To Reproduce

1. Build the application next build
2. Start the application next start
3. Navigate to category 1
4. Navigate to category 2 with hard refresh

Current vs. Expected behavior

Current Behavior

Product 1 appears in category 1 and category 2

Currently they return return identical data, but have a different rsc hash

Expected Behaviour

If the data is the same there should be only one rsc hash.

With a high throughput global ecommerce site I want to cache identical data close to the customer in a CDN. The different rsc hashes mean that I will get CDN cache misses and traffic will have to go back to the server, potentially a distance from the customer and with a slower response time.

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:42 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6000
  Available memory (MB): 65536
  Available CPU cores: 10
Binaries:
  Node: 18.18.0
  npm: 10.1.0
  Yarn: 1.22.19
  pnpm: N/A
Relevant Packages:
  next: 14.2.3 // Latest available version is detected (14.2.3).
  eslint-config-next: 14.1.0
  react: 18.3.1
  react-dom: 18.3.1
  typescript: 5.4.5
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

Performance

Which stage(s) are affected? (Select all that apply)

next start (local)

Additional context

Hi I'm trying to work out how I make the RSC requests work with a CDN when self hosting Nextjs. Is there any more info on this subject. I'm working on an ecommerce site with 100,000 products. Those products could appear in numerous product listing pages (search result pages). Each of those PLP pages are given unique URLs for SEO purposes. Take for example the following URLs:

/mens/
/mens/trainers
/mens/trainers/brand
/mens/trainers/brand?facet-price=%3A168

They could all have the same products in. Due to the way the rsc hash is calculated that means I get a different _rsc params on each listing page, even though the contents of that response is exactly the same.

product/299336/?_rsc=1vl30
product/299336/?_rsc=qe3go
product/299336/?_rsc=1vg99
product/299336/?_rsc=1stsw

I have even gone to different areas of the site cart, checkout, order history, all with links back to the same product, they each produce different _rsc params, but still the data returned is identical?

On a high throughput site I want to be able to cache identical data in the CDN close to the customer. At the moment that would be impossible as there would be to many variations on the rsc hash to make the caching effective.

For solutions I think I only have 2 options:

Cache all rsc requests in the CDN - this would end up caching loads of duplicate data and get cache misses when they should be hits
Pass all request through to the Nextjs server. - With this solution I would worry the server would be overloaded at peak periods.
In both cases there would be an extra cost to the client

I'm trying to understand why the rsc has is different when it is always returning identical data? What is the purpose of this hash? As mentioned before could this just be set to _rsc=1? We would also have issues with the Vary header as it is currently returned like this:

Vary: RSC, Next-Router-State-Tree, Next-Router-Prefetch, Next-Url

In this case the Next-Router-State-Tree and Next-Url will be set on where you are coming from and do not necessarily have an impact on the data need for the page we are going to. The Vary header will again have an impact on the CDN

This issue has been rasied in the following discussion thread https://github.com/vercel/next.js/discussions/59167

_NEXT-3327

Start with the linked CodeSandbox and reproduce the behavior using next build and next start, comparing the RSC hashes and Vary headers across the listed navigation paths. Read the linked discussion for prior context and investigate how the RSC request inputs affect caching. Done should include a confirmed explanation and an agreed resolution for effective CDN caching.