Certain recursive types cause "Maximum call stack size exceeded" error in TS server

### 🔎 Search Terms

visitType, visitObjectType, visitSymbol, RangeError, TypeScript Server Error

### 🕗 Version & Regression Information

- This is the behavior in versions 5.9.2 and 6.0.0-dev.20250909

### ⏯ Playground Link

_No response_

### 💻 Code

I'm not sure how to reproduce this at low level. I met this error in VS Code when it stopped showing "Extract" and "Move" groups in the "Refactor..." menu. Then I looked in the extension logs and saw an error in TS Server.

Steps to reproduce are:

1. In an empty TypeScript project (tsconfig.json may be nonexistent), create a .ts file with:
```ts
function func(param: ProblematicType) {}

type ProblematicType = {
prop: ProblematicType;
};

class TestRefactoring {
createElement() {
document.createElement('span');
}
}
```
2. Select `document.createElement('span');`
3. Make a right click on the selection
4. Click "Refactor..."

### 🙁 Actual behavior

1. "Refactor..." menu:

3. Extension Host logs in VS Code:
```
[error] [vscode.typescript-language-features] provider FAILED
[error] Error: TypeScript Server Error (6.0.0-dev.20250909)
Maximum call stack size exceeded
RangeError: Maximum call stack size exceeded
at instantiateType (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:68177:27)
at instantiateTypeWorker (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:68263:9)
at instantiateTypeWithAlias (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:68210:20)
at instantiateType (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:68178:29)
at getMappedType (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:67858:67)
at \.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:67987:56
at map (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:2492:19)
at getObjectTypeInstantiation (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:67987:29)
at instantiateTypeWorker (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:68235:16)
at instantiateTypeWithAlias (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:68210:20)
at instantiateType (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:68178:29)
at getTypeOfInstantiatedSymbol (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:61397:40)
at getTypeOfSymbol (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:61461:14)
at visitSymbol (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:49811:17)
at visitObjectType (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:49796:9)
at visitType (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:49725:11)
[...]
at visitSymbol (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:49812:7)
at visitObjectType (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:49796:9)
at visitType (\.vscode\extensions\ms-vscode.vscode-typescript-next-6.0.20250909\node_modules\typescript\lib\typescript.js:49725:11)
at n.create (\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\typescript-language-features\dist\extension.js:2:357355)
at y.dispatchResponse (\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\typescript-language-features\dist\extension.js:2:30813)
at y.dispatchMessage (\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\typescript-language-features\dist\extension.js:2:29254)
at ChildProcess. (\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\typescript-language-features\dist\extension.js:2:28539)
at ChildProcess.emit (node:events:518:28)
at ChildProcess.emit (node:domain:489:12)
at emit (node:internal/child_process:949:14)
at processTicksAndRejections (node:internal/process/task_queues:91:21)
```

### 🙂 Expected behavior

1. "Refactor..." menu:

3. No error.
### Additional information about the issue

The type `ProblematicType` might not make sense to you, but it's a simplified (made as simple as I was able to) example of a real type in the @types/oojs library where I posted the issue as well, see https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/73718.

For some reason, in VS Code this error appears only when the class is generic and an out of scope entity is referenced in the selection:
- If you remove the generic parameter from `TestRefactoring`, the error will be gone.
- If you remove the `document` reference from `createElement()` and replace it with `smth`, the error will be gone.

Start with the minimal TypeScript reproduction and the TS Server refactor request described in the issue. Use the reported stack around instantiateType, getMappedType, visitSymbol, visitObjectType, and visitType to trace the recursive path. Done means the example no longer causes a RangeError and the expected Refactor menu is available.

Written by the indexing model from the issue text.