Lazily loaded plugin loads before filetype is set by another plugin

By default, [Vim thinks that .jl files are librep Lisp](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=71279). The Julia language uses .jl as its file extension, and you can get .jl files to be recognized as Julia with the [julia-vim](https://github.com/JuliaEditorSupport/julia-vim) plugin. However, it seems that lazily loaded plugins with `{ 'for': 'lisp' }` load for .jl files, even with julia-vim installed with its `Plug` line before that of the other. That suggests to me a load order issue, like #85, but that particular issue was closed years ago.

For reference, here is the specific setup where this is occurring for me:
```vim
call plug#begin('~/.local/share/nvim/plugged')
" ...
Plug 'JuliaEditorSupport/julia-vim'
" ...
Plug 'eraserhd/parinfer-rust', { 'for': ['scheme', 'lisp', 'clojure', 'racket'], 'do': 'cargo build --release' }
call plug#end()
```

In the meantime I've been able to get around this by setting
```vim
autocmd FileType julia let g:parinfer_enabled=0
```
but that sort of defeats the purpose of lazily loading parinfer-rust in the first place.

------------------------------

```
:version
NVIM v0.4.3
Build type: Release
LuaJIT 2.0.5
Compilation: /usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -DNDEBUG -DMIN_LOG_LEVEL=3 -Wall -Wextra -pedantic
-Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wmissing-prototypes -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fdiagnos
tics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -I/tmp/neovim-20191107-13403-1or2rj3/neovim-
0.4.3/build/config -I/tmp/neovim-20191107-13403-1or2rj3/neovim-0.4.3/src -I/usr/local/include -I/tmp/neovim-20191107-13403-1or2rj3/neovim-0.4.3/deps-build/inclu
de -I/usr/local/opt/gettext/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include -I/tmp/neo
vim-20191107-13403-1or2rj3/neovim-0.4.3/build/src/nvim/auto -I/tmp/neovim-20191107-13403-1or2rj3/neovim-0.4.3/build/include
Compiled by brew@Mojave.local

Features: +acl +iconv +tui
See ":help feature-compile"

system vimrc file: "$VIM/sysinit.vim"
fall-back for $VIM: "/usr/local/Cellar/neovim/0.4.3/share/nvim"
```

- Type:
- [x] Bug
- [ ] Enhancement
- [ ] Feature Request
- [x] Question
- OS:
- [ ] All/Other
- [ ] Linux
- [x] OS X
- [ ] Windows
- Vim:
- [ ] Terminal Vim
- [ ] GVim
- [x] Neovim