Selected region changes after pressing ctrl-b twice

Describe the bug
The boundary of a selected region would change by pressing ctrl-b twice to scroll two pages up.

To Reproduce
Steps to reproduce the behavior:

1. Prepare a file with sufficient length.
2. G: go to the end of the file.
3. V: select the current line.
4. ctrl-b ctrl-b
5. Scroll down with a mouse and you will see that the selection boundary changes and is not at the end of the file.

Expected behavior
Vim and Neovim would preserve the correct selection range, which is from the current cursor position to the end of the file.

Environment (please complete the following information):

• Extension (VsCodeVim) version: v1.8.1
• VSCode version: 1.35.1
• OS: macOS 10.14

Additional context
This bug only occurs when ctrl-b is used in visual mode, ctrl-f is not affected.
The input sequence ctrl-b followed by j or k also results in strange behaviour.

No file, test, or code entry point is named. First reproduce the sequence in the issue—visual selection, G, ctrl-b twice, then mouse scrolling—and inspect the visual-mode handling involved; done means the selection remains from the cursor position to the end of the file, including after ctrl-b followed by j or k.