Very high memory usage (>10GB) on a specific project

Type: Bug

I am not exactly sure on how to reproduce the problem.
When I open VSCode with this folder, the memory usage goes crazy. From Windows Task Manager I can see it can reach >13GB with the majority coming from Microsoft.CodeAnalysis.LanguageServer

When I ran the ProcessExplorer tool, this memory usage does not match what's being reported on Task Manager, it is significantly lower. Followed some of the steps here: https://github.com/Microsoft/vscode/wiki/Performance-Issues and ran Extension Bisect and it isolated this extension.

Previously I had installed the new Unity extension (VisualStudioToolsForUnity.vstuc) along with it's dependencies, but noticed this problem started happening. I disabled and uninstalled it but the problem did not go away so I am not too sure what happened after that.

Extension version: 1.7.0
VS Code version: Code 1.81.1 (6c3e3dba23e8fadc360aed75ce363ba185c49794, 2023-08-09T22:22:42.175Z)
OS version: Windows_NT x64 10.0.19045
Modes:

System Info

|Item|Value|
|---|---|
|CPUs|AMD Ryzen 7 4800H with Radeon Graphics (16 x 2895)|
|GPU Status|2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled|
|Load (avg)|undefined|
|Memory (System)|31.42GB (14.71GB free)|
|Process Argv|--crash-reporter-id d9402af1-264c-402b-ad11-b3ec02ddb916|
|Screen Reader|no|
|VM|0%|

A/B Experiments

```
vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931:30280409
vshan820:30294714
vstes263:30335439
vscorecescf:30445987
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
pythonvs932:30410667
py29gd2263:30792226
vsclangdc:30486549
c4g48928:30535728
dsvsc012:30540252
pynewext54:30695312
azure-dev_surveyone:30548225
vsccc:30803844
2e4cg342:30602488
89544117:30613380
a9j8j154:30646983
showlangstatbar:30737416
03d35959:30757346
pythonfmttext:30731395
pythoncmv:30756943
fixshowwlkth:30771522
showindicator:30805244
pythongtdpath:30769146
i26e3531:30792625
gsofb:30804716
pythonnosmt12:30797651
pythonidxpt:30805730
pythonnoceb:30805159
dsvsc013:30795093
dsvsc014:30804076
diffeditorv2:30812749

```