Code server disconnects after browser inactivity for a while

### Is there an existing issue for this?

- [X] I have searched the existing issues

### OS/Web Information

- Web Browser: Chrome, Firefox
- Local OS: macOS, Windows
- Remote OS: Amazon linux 2 (Sagemaker)
- Remote Architecture: x86_64 GNU/Linux
- `code-server --version`: code-server: v4.16.1, Code: 1.80.2

### Steps to Reproduce

1. launch code server on Sagemaker notebook instances
2. Connect from local machine using Chrome, or Firefox
3. When the go computer is idle (e.g. in screen saver mode) for a while (the duration is not consistent, but 15min should do), or when the browser is out of focus for a long time, the vscode client page gets disconnected: the file system is no longer responsive (we have a blue clock icon and a horizontal line that scrolls continuously, see screeshot); and the terminal no longer respond also
4. Refreshing the browser page brings the connection back to normal

### Expected

The vscode code client should not be disconnected when the computer has access to internet

### Actual

Vscode client page gets disconneted when computer is idle or when browser is out of focus for a while

### Logs

_No response_

### Screenshot/Video


### Does this bug reproduce in native VS Code?

I did not test native VS Code

### Does this bug reproduce in GitHub Codespaces?

I did not test GitHub Codespaces

### Are you accessing code-server over a secure context?

- [ ] I am using a secure context.

### Notes

_No response_