[BUG] Salt-master 3000.3, git_pillar and empty pillar issue

Description
After upgrading our production system's salt-master from 2019.2.4 to 3000.3, ext-pillars from git over https are working somewhat unreliable. When master pillar cache is empty and one runs 'salt-call pillar.items' on minion, master does not fetch git branch content immediately, instead master returns empty pillars to minion. Following error appears to master log "[salt.utils.gitfs :1593][ERROR ][23305] Failed to checkout testing from git_pillar remote 'testing https://git.our-internal-git.foo/git/salt': remote ref does not exist".
After some time (usually minutes, but sometimes might take hours), master fetches git branch succesfully and pillar cache is as it should be.

In production we use pygit2+python2 (both from CentOS repo). In test system (RHEL7) we were able to reproduce the issue with following different configurations
salt-master 3000.3 + python2 + pygit2 0.26.2 (from RHEL repo)
salt-master 3000.3 + python2 + gitpython (from RHEL repo)
salt-master 3000.3 + python3 + pygit2 1.0.2
salt-master 3001.1 + python3 + pygit2 1.0.2

Downgrade salt-master back to 2019.2.4 fixes the issue instantly.

Setup

ext_pillar:
  - git:
    - testing https://git.our-internal-git.foo/git/salt:
      - env: base
      - root: somedirectory
      - user: foo
      - password: bar
      - ssl_verify: False

Steps to Reproduce the behavior

1. Install fresh CentOS 7 (or RHEL7) + Salt-master and minion 3000.3

2. Install pygit2 (or gitpython) from CentOS repo

3. Setup ext_pillars as above

4. Start salt-master and minion

5. Run 'salt-call pillar.items' -> returns empty pillars and error message appears in master log and /var/cache/salt/master/git_pillar does not include full clone of git branch

6. Wait some time, run salt-call pillar.items again

7. Pillars are working as expected and /var/cache/salt/master/git_pillar contains all files as expected

8. To reproduce the issue again, just rm -rf /var/cache/salt/master/git_pillar/* && systemctl restart salt-master

Versions Report

Salt Version:
           Salt: 3000.3

Dependency Versions:
           cffi: 1.6.0
       cherrypy: Not Installed
       dateutil: 1.5
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.7.2
        libgit2: 0.26.3
       M2Crypto: 0.31.0
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 2.7.5 (default, Mar 20 2020, 17:08:22)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.3.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.1.4

System Versions:
           dist: redhat 7.9 Maipo
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-1160.el7.x86_64
         system: Linux
        version: Red Hat Enterprise Linux Server 7.9 Maipo

Start with the git_pillar flow in salt.utils.gitfs, especially the error reported around line 1593, and reproduce it using the CentOS/RHEL 7 setup and ext_pillar configuration shown. Compare the first and later runs after clearing /var/cache/salt/master/git_pillar; done means the initial salt-call pillar.items fetches the branch and returns its pillars without the remote-ref error.