Initializing an instance of salt.client.Caller() causes the minion to freeze

Hello,

When I initialize an instance of salt.client.Caller() inside a custom grain, the minion freezes immediately after a sync_grains. e.g.:

caller = salt.client.Caller()

This looks identical to the issue that was apparently resolved here:

https://github.com/saltstack/salt/issues/28091

This is a pretty major blocker for me right now. Any ideas would be appreciated.

Thanks,
Oliver

Log on minion shows many of these:

2020-02-17 13:22:02,878 [salt.loader      :817 ][CRITICAL][5388] Failed to load grains defined in grain file set_device_type.set_device_type in function <function set_device_type at 0x000000000824AF28>, error:
Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt\loader.py", line 810, in grains
    ret = funcs[key](**kwargs)
  File "c:\salt\var\cache\salt\minion\extmods\grains\set_device_type.py", line 35, in set_device_type
    caller = salt.client.Caller()
  File "c:\salt\bin\lib\site-packages\salt\client\__init__.py", line 1992, in __init__
    self.sminion = salt.minion.SMinion(self.opts)
  File "c:\salt\bin\lib\site-packages\salt\minion.py", line 844, in __init__
    lambda: self.eval_master(self.opts, failed=True)
  File "c:\salt\bin\lib\site-packages\tornado\ioloop.py", line 453, in run_sync
    self.start()
  File "c:\salt\bin\lib\site-packages\tornado\ioloop.py", line 755, in start
    raise RuntimeError("IOLoop is already running")
RuntimeError: IOLoop is already running

This is a Windows minion. I can confirm it also happens on 2018.3.3:

c:\salt\var\log\salt>salt-call --versions-report
Salt Version:
           Salt: 2019.2.2

Dependency Versions:
           cffi: 1.12.2
       cherrypy: 17.4.1
       dateutil: 2.8.0
      docker-py: Not Installed
          gitdb: 2.0.6
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10.1
        libgit2: Not Installed
        libnacl: 1.6.1
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: Not Installed
   pycryptodome: 3.8.1
         pygit2: Not Installed
         Python: 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)]
   python-gnupg: 0.4.4
         PyYAML: 3.13
          PyZMQ: 18.0.1
           RAET: Not Installed
          smmap: 2.0.5
        timelib: 0.2.4
        Tornado: 4.5.3
            ZMQ: 4.3.1

System Versions:
           dist:
         locale: cp1252
        machine: AMD64
        release: 10
         system: Windows
        version: 10 10.0.18362  Multiprocessor Free

From the master:

Salt Version:
           Salt: 2019.2.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Oct  8 2019, 14:14:10)
   python-gnupg: 0.3.8
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-62-generic
         system: Linux
        version: Ubuntu 16.04 xenial

Start with the Caller constructor in salt/client/init.py and the SMinion initialization in salt/minion.py, using the set_device_type.py custom grain and the reported Windows traceback to reproduce the failure. Done means initializing salt.client.Caller() from a grain no longer leaves the minion frozen or raises “IOLoop is already running”.