[BUG] salt-master unable to connect gitfs of ext_pillar

Description
I'am rebuilding my salt-master and configuring gitfs and ext_pillar git but gitfs is unable to connect to the git repo.

The ssh keys are working and have been tested using ssh client as does a git clone on the server with these keys.

The salt master config has been used on the original server without problems, only the keys are renewed, i have tested ED25519 and RSA type keys.

Setup

fileserver_backend:
  - roots
  - gitfs

gitfs_provider: pygit2

gitfs_remotes:
  - git@git.mydomain.coml:saltstack/salt-gitfs-remote-enviroments.git:
    - pubkey: /usr/local/etc/salt/ssh_keys/saltstack-repo_id_rsa.pub
    - privkey: /usr/local/etc/salt/ssh_keys/saltstack-repo_id_rsa
    - root: states
    - saltenv:
      - base:
        - ref: master

ext_pillar:
  - git:
    - master git@git.newimage.nl:saltstack/salt-gitfs-remote-enviroments.git:
      - pubkey: /usr/local/etc/salt/ssh_keys/saltstack-repo_id_rsa.pub
      - privkey: /usr/local/etc/salt/ssh_keys/saltstack-repo_id_rsa
      - root: pillar
      - env: base

Steps to Reproduce the behavior
Start the salt-master , maybe in debug mode and check log of output.

Expected behavior
Connection to Git

Screenshots

[ERROR   ] Error occurred fetching gitfs remote 'git@git.mydomain.com:saltstack/salt-gitfs-remote-enviroments.git': Failed to retrieve list of SSH authentication methods: Failed getting response
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1741, in _fetch
    fetch_results = origin.fetch(**fetch_kwargs)
  File "/usr/local/lib/python3.7/site-packages/pygit2/remote.py", line 423, in fetch
    check_error(err)
  File "/usr/local/lib/python3.7/site-packages/pygit2/errors.py", line 61, in check_error
    raise GitError(message)
_pygit2.GitError: Failed to retrieve list of SSH authentication methods: Failed getting response

[DEBUG   ] Set update lock for git_pillar remote 'master git@git.newimage.nl:saltstack/salt-gitfs-remote-enviroments.git'
[DEBUG   ] Fetching git_pillar remote 'master git@git.newimage.nl:saltstack/salt-gitfs-remote-enviroments.git'
[ERROR   ] Error occurred fetching git_pillar remote 'master git@git.newimage.nl:saltstack/salt-gitfs-remote-enviroments.git': Failed to retrieve list of SSH authentication methods: Failed getting response
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1741, in _fetch
    fetch_results = origin.fetch(**fetch_kwargs)
  File "/usr/local/lib/python3.7/site-packages/pygit2/remote.py", line 423, in fetch
    check_error(err)
  File "/usr/local/lib/python3.7/site-packages/pygit2/errors.py", line 61, in check_error
    raise GitError(message)
_pygit2.GitError: Failed to retrieve list of SSH authentication methods: Failed getting response
[DEBUG   ] Removed update lock for git_pillar remote 'master git@git.mydomain.com:saltstack/salt-gitfs-remote-enviroments.git'

Versions Report

Salt Version:
           Salt: 2019.2.4
 
Dependency Versions:
           cffi: 1.14.0
       cherrypy: unknown
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10.1
        libgit2: 1.0.0
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 1.2.0
         Python: 3.7.7 (default, Apr 12 2020, 21:25:06)
   python-gnupg: Not Installed
         PyYAML: 5.3.1
          PyZMQ: 19.0.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.1
 
System Versions:
           dist:   
         locale: UTF-8
        machine: amd64
        release: 12.1-RELEASE-p3
         system: FreeBSD
        version: Not Installed

Used relevant OS Packages
git-lite-2.26.2                Distributed source code management tool (lite package)
libgit2-1.0.0                  Portable, pure C implementation of the Git core
py37-pygit2-1.2.0_1            Python bindings for libgit2
libssh2-1.8.2,3                Library implementing the SSH2 protocol

Additional context

Reproduce the failure by starting salt-master in debug mode and inspect the traceback path in salt/utils/gitfs.py, especially _fetch and its pygit2/libgit2 SSH interaction. Compare the configured keys and direct git clone behavior with the reported pygit2 1.2.0, libgit2 1.0.0, and libssh2 versions; done means both gitfs and git_pillar fetch their remotes successfully.