🐛 BUG: Workers TTFB slows down based on unused dynamic imports (additional modules)

### Which Cloudflare product(s) does this pertain to?

Pages, Workers Runtime, Wrangler core

### What version(s) of the tool(s) are you using?

wrangler 3.38.0

### What version of Node are you using?

18.18.2

### What operating system and version are you using?

macOS Sonoma 14.2.1

### Describe the Bug

### Observed behavior
Additional modules that are used as dynamic imports and not bundled will still slow down Workers TTFB if they're uploaded, even if they are never effectively dynamically imported. This can be replicated on both Workers and Pages (with `_worker.js/ dir`), and whilst it affects both, it seems quite worse on Pages.

### Expected behavior
Without knowing any internal infrastructure limits, ideally dynamic imports would be lazily instantiated/parsed/exec in a way that mitigates worsening performance on the Worker as a whole except when necessary.

This is a pattern that most JS frameworks deployed to Cloudflare Pages are relying on (see official Next.js adapter [next on pages](https://github.com/cloudflare/next-on-pages/blob/main/docs/technical/lazy-loading.md), Nuxt, Remix, etc).

### Steps to reproduce

You can clone both an example using Worker with `--no-bundle` and an example using Pages with `_worker.js/ dir` below. In both cases, you can see the previous commits, but just by modifying the files you include inside the `lazy` folder you can ultimately impact the performance of the Worker when deployed (couldn't measure any differences locally).

### Please provide a link to a minimal reproduction

- Pages with `_worker.js/ dir`: https://github.com/juanferreras/dynamic-import-cold-start-pages
- Worker with `--no-bundle`: https://github.com/juanferreras/dynamic-import-cold-start-workers

See the `README.md` for instructions and sizes of each of the files. The setup is the exact same on both projects, we have a simple Worker that dynamically imports `.js` files with a different volumes of data.

The path we're using to compare is always NOT dynamically importing the heavy dependencies but the change is whether those files are simply included as additional modules when deploying or not.

### Please provide any relevant error logs

N/A

## Benchmarks

Using WebPageTest Virginia - EC2 Motorola G Power with 4G, running 9 tests for each (first view only) and taking into account the median.

### Pages with `_worker.js/ dir`

The following 4 branches have been deployed.
1. https://small.dynamic-import-cold-start-pages.pages.dev/ - [WebPageTest Link](https://www.webpagetest.org/result/240327_BiDcTF_5JV/?test=240327_BiDcTF_5JV&medianMetric=TTFB)
2. https://medium.dynamic-import-cold-start-pages.pages.dev/ - [WebPageTest Link](https://www.webpagetest.org/result/240327_AiDc9J_5Q8/?test=240327_AiDc9J_5Q8&medianMetric=TTFB)
3. https://large.dynamic-import-cold-start-pages.pages.dev/ - [WebPageTest Link](https://www.webpagetest.org/result/240327_AiDcEQ_5QB/?test=240327_AiDcEQ_5QB&medianMetric=TTFB)
4. https://titanic.dynamic-import-cold-start-pages.pages.dev/ - [WebPageTest Link](https://www.webpagetest.org/result/240327_BiDcPR_5K1/?test=240327_BiDcPR_5K1&medianMetric=TTFB)
- [WebPageTest comparison](https://www.webpagetest.org/video/compare.php?tests=240327_BiDcTF_5JV,240327_AiDc9J_5Q8,240327_AiDcEQ_5QB,240327_BiDcPR_5K1)
- [WebPageTest Plot](https://www.webpagetest.org/graph_page_data.php?tests=240327_BiDcTF_5JV,240327_AiDc9J_5Q8,240327_AiDcEQ_5QB,240327_BiDcPR_5K1&medianMetric=SpeedIndex&fv=1&median_value=1&control=0)
- Time to First Byte (TTFB) comparison:

**Comment**: the impact is a very noticeable even with just adding `medium` (e.g. way before totalling 1MB gzipped code). It continues growing linearly (note the `titanic` version is still only using 3 MB gzipped out of a total of 10 MB in theory allowed for the platform when using paid plans).

### Worker with `--no-bundle`

The following 4 workers have been deployed.
1. https://dynamic-import-cold-start-workers.juanmf.workers.dev/ - [WebPageTest link](https://www.webpagetest.org/result/240327_BiDc86_5K2/?test=240327_BiDc86_5K2&medianMetric=TTFB)
2. https://dynamic-import-cold-start-workers-medium.juanmf.workers.dev/ - [WebPageTest link](https://www.webpagetest.org/result/240327_AiDc57_5QM/?test=240327_AiDc57_5QM&medianMetric=TTFB)
3. https://dynamic-import-cold-start-workers-large.juanmf.workers.dev/ - [WebPageTest link](https://www.webpagetest.org/result/240327_BiDc4V_5K9/?test=240327_BiDc4V_5K9&medianMetric=TTFB)
4. https://dynamic-import-cold-start-workers-titanic.juanmf.workers.dev/ - [WebPageTest link](https://www.webpagetest.org/result/240327_AiDcDB_5QN/?test=240327_AiDcDB_5QN&medianMetric=TTFB)
- [WebPageTest comparison](https://www.webpagetest.org/video/compare.php?tests=240327_BiDc86_5K2,240327_AiDc57_5QM,240327_BiDc4V_5K9,240327_AiDcDB_5QN)
- [WebPageTest Plot](https://www.webpagetest.org/graph_page_data.php?tests=240327_BiDc86_5K2,240327_AiDc57_5QM,240327_BiDc4V_5K9,240327_AiDcDB_5QN&medianMetric=SpeedIndex&fv=1&median_value=1&control=0)
- Time to First Byte (TTFB) comparison:

**Comment**: it seems a lot less directly impacted, when adding `medium` it does not change, and when adding the larger ones it's affected but less than the Pages ones (e.g. like `log(n)` instead of linearly).

---

Please do let me know if I have any errors on the repositories and/or if there are ideas worth trying to see if (a) we could get the Pages one to perform similar to the Worker, and/or (b) we can do dynamic imports slightly different to prevent the overall performance impact