Listing SSH targets is slow with big config/many hosts

Version: 1.45.0-insider
Commit: 0778354f3a6a2c21b7f738a5e6b02e2f1c765e73
Date: 2020-04-27T05:34:13.940Z
Electron: 7.2.2
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Linux x64 5.6.6-200.fc31.x86_64

• VSCode Version: 1.45-insider
• Local OS Version: Fedora 31
• Remote OS Version: N/A
• Remote Extension/Connection Type: SSH

Steps to Reproduce:

1. Create large (I have 36K lines and 7K hosts) ~/.ssh/config file with hosts

2. Open SSH targets tab of vscode

This is slow and takes a lot of CPU, the console outputs this:
WARN UNRESPONSIVE extension host, 'ms-vscode-remote.remote-ssh' took 93% of 5265.002ms, saved PROFILE here: '/tmp/exthost-7388f2.cpuprofile' (the profile contains internal data so I'd rather not upload it, but it should be easy to reproduce with dummy hosts).

Does this issue occur when you try this locally?: N/A
Does this issue occur when you try this locally and all extensions are disabled?: N/A

Kind of related, it would be good to have a filtering mechanism for when you have many hosts.