Can't switch to Buffer if viewing a help file which takes up the whole window

- [x] I have read through https://github.com/junegunn/fzf.vim/blob/master/README.md
- [x] I have read through https://github.com/junegunn/fzf/blob/master/README-VIM.md
- [x] I have read through the manual page of fzf (`man fzf`)
- [x] I have searched through the existing issues

Before submitting
=================
- [x] Make sure that you have the latest version of fzf and fzf.vim
- [x] Check if your problem is reproducible with a minimal configuration

Start Vim with a minimal configuration
======================================

# Steps to Reprodue:
1. Start vim:
```
vim -Nu <(curl https://gist.githubusercontent.com/junegunn/6936bf79fedd3a079aeb1dd2f3c81ef5/raw)
```
2. Open a file.
3. `:h` to bring up the help
4. In the help window press `^W o` to make it the only window.
5. `:Buffer` and try switch back to the file you originally opened
6. FZF won't let you select it the file.

I am guessing it is because it is the current file, and help buffers are maybe treated a little differently. Its confusing when consulting the help a lot, one has to first think, am I switch from a help screen or from a normal buffer.