Cannot connect to Apollo dev tools from Apollo client 3.9 or 3.11

window.__APOLLO_CLIENT__ is undefined

We can see if we log out the window object that there are references to the Apollo client and the Apollo dev tools but it doesn't seem to have initialised correctly

### `@apollo/client` version
We have tried with both 3.9 and 3.11 which we recently upgraded to
We've also tried the new devtools config in v3.11 and the VSCode extension but it looks like the same problem in all

### Apollo Client Devtools version
4.19.5 (Chrome)

Might be related to https://github.com/apollographql/apollo-client-devtools/issues/1580

in 3.11 I couldn't get it to connect at all, in 3.9 it seemed to connect at first but then dropped the connection

Start by reproducing the missing window.__APOLLO_CLIENT__ reference with Apollo Client 3.9 and 3.11, Apollo Client Devtools 4.19.5, and the reported devtools configurations. Compare the initial connection and subsequent disconnect behavior, including the VSCode extension, and identify why the client is not initialized or the connection drops; done means a reproducible fix with a verified connection.

Written by the indexing model from the issue text.