[BUG] 'OrderedDict' object has no attribute 'startswith' when using exclude with an id

Description

Two id's in an sls, one that creates a file and two that has a requisite (onchanges) on that file.

When you exclude the id: one, an Attribute error is raised:

File "/usr/lib/python3/dist-packages/salt/state.py", line 1810, in
for state in [_state for _state in iter(high[_id]) if not _state.startswith("__")]:
AttributeError: 'OrderedDict' object has no attribute 'startswith'

This can be tested from the cli:

salt 'myhost' state.apply exclude="[{'id': 'id_to_exclude'}]" test=true

See: https://docs.saltstack.com/en/master/ref/modules/all/salt.modules.state.html

Setup

docker.sls

# Manage the unit file to setup iptables which protects docker containers to be reached from the network.
systemd-docker-service-iptables:
  file.managed:
    - name: {{ unitdir }}/{{ unitfile }}
    - user: root
    - group: root
    - mode: 644
    - dir_mode: 755
    - makedirs: True
    - contents: |
        [Service]
        ExecStartPost=/sbin/iptables -I DOCKER 1 -i {{ host.if_main }} -j DROP

# restart systemd daemon on changes
docker-restart:
  cmd.run:
    - name: /bin/systemctl daemon-reload && /bin/systemctl restart docker.service
    - onchanges:
      - file: {{ unitdir }}/{{ unitfile }}

myhost.sls

include:
  - common.some.state

exclude:
  -  systemd-docker-service-iptables

Steps to Reproduce the behavior

salt 'myhost' state.apply exclude="[{'id': 'systemd-docker-service-iptables'}]" test=true

myhost:
    The minion function caused an exception: Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/minion.py", line 1869, in _thread_return
        return_data = minion_instance.executors[fname](
      File "/usr/lib/python3/dist-packages/salt/executors/direct_call.py", line 12, in execute
        return func(*args, **kwargs)
      File "/usr/lib/python3/dist-packages/salt/modules/state.py", line 801, in apply_
        return highstate(**kwargs)
      File "/usr/lib/python3/dist-packages/salt/modules/state.py", line 1123, in highstate
        ret = st_.call_highstate(
      File "/usr/lib/python3/dist-packages/salt/state.py", line 4464, in call_highstate
        return self.state.call_high(high, orchestration_jid)
      File "/usr/lib/python3/dist-packages/salt/state.py", line 3192, in call_high
        high, req_in_errors = self.requisite_in(high)
      File "/usr/lib/python3/dist-packages/salt/state.py", line 1810, in requisite_in
        for state in [_state for _state in iter(high[_id]) if not _state.startswith("__")]:
      File "/usr/lib/python3/dist-packages/salt/state.py", line 1810, in <listcomp>
        for state in [_state for _state in iter(high[_id]) if not _state.startswith("__")]:
    AttributeError: 'OrderedDict' object has no attribute 'startswith'

Expected behavior

The highstate would complete normally and indicate which states/id's failed.

<snip>
----------
          ID: docker-restart
    Function: cmd.run
        Name: /bin/systemctl daemon-reload && /bin/systemctl restart docker.service
      Result: False
     Comment: The following requisites were not found:
                                 onchanges:
                                     file: /etc/systemd/system/docker.service.d/block-external-access.conf
     Started: 11:21:56.707274
    Duration: 0.015 ms
     Changes:

<snip>
Summary for myhost
-------------
Succeeded: 66 (unchanged=1)
Failed:     1
-------------
Total states run:     67
Total run time:   28.343 s

Versions Report

Observations: minion runs python 3.8, master 3.6 but that should not have a play in this.

minion myhost:

Salt Version:
           Salt: 3001

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.7.3
      docker-py: 4.2.2
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.10.1
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: Not Installed
   pycryptodome: 3.6.1
         pygit2: Not Installed
         Python: 3.8.2 (default, Apr 27 2020, 15:53:34)
   python-gnupg: 0.4.5
         PyYAML: 5.3.1
          PyZMQ: 18.1.1
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.2

System Versions:
           dist: ubuntu 20.04 focal
         locale: utf-8
        machine: x86_64
        release: 5.4.0-40-generic
         system: Linux
        version: Ubuntu 20.04 focal

salt-master:

Salt Version:
           Salt: 3001

Dependency Versions:
           cffi: 1.12.3
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: 4.2.2
          gitdb: 2.0.3
      gitpython: 2.1.8
         Jinja2: 2.10
        libgit2: 0.26.0
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: 3.4.7
         pygit2: 0.26.2
         Python: 3.6.9 (default, Apr 18 2020, 01:56:04)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 17.1.2
          smmap: 2.0.3
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: ubuntu 18.04 Bionic Beaver
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-109-generic
         system: Linux
        version: Ubuntu 18.04 Bionic Beaver

Additional context

I've created a fix that allows the state to report failed states in salt/state.py starting at line 1810:

                                           for state in iter(high[_id]):
                                                # When a requisite is tied to an excluded id, state is an ordered dict.
                                                if hasattr(state, 'startswith'):
                                                    if not state.startswith("__"):
                                                        for j in iter(high[_id][state]):
                                                            if (isinstance(j, dict) and "name" in j):
                                                                if j["name"] == ind:
                                                                    ind = {state: _id}
                                                                    found = True

So far I have not ran into troubles with this change.

.. Note:: Using the construct below did not seem to work:

if isinstance(state, str) and not state.startswith("___"):
    for j ....

Pull-request can be made.

Start in salt/state.py around line 1810 and reproduce the issue with the provided state.apply exclude command. Inspect how requisite_in handles entries in high[_id] when an excluded id produces an OrderedDict. Done means the highstate no longer raises AttributeError and reports the affected failed state as shown in the expected output.