saltstack / saltstack/salt

[BUG] minion returns success but orchestrate runner returns fail

Open
#58,739 9 comments 0 reactions 2 assignees View on GitHub

@twangboy is already working on this.

Since Feb 9, 2021.

bug Confirmed severity-medium Windows
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Minion logs return 'success': True, but orchestrate returns failure.
Also a related issue where state.apply is successful while state.orch fails.

Minion logs:

[salt.loaded.int.module.cmdmod:414 ][INFO    ][2072] Executing command 'Powershell -NonInteractive -NoProfile "Start-Process -Wait -FilePath \"C:\windows\TEMP\vs_BuildTools.exe\" -ArgumentList \"--passive\""' in directory 'C:\windows\system32\config\systemprofile'
[salt.minion      :1999][DEBUG   ][4240] minion return: {'success': True, 'return': {'pid': 2072, 'fun': 'state.sls', 'arg': ['states.scratch.first', {'__kwarg__': True, 'queue': False, 'concurrent': False}], 'tgt': 'salt-minion-new', 'jid': '20201015223200590679', 'ret': '', 'tgt_type': 'glob', 'user': 'sudo_azureuser'}, 'retcode': 0, 'jid': '20201015223205683741', 'fun': 'saltutil.find_job', 'fun_args': ['20201015223200590679']}
[salt.minion      :1999][DEBUG   ][2072] minion return: {'success': True, 'return': {'module_|-log first_|-log.info_|-run': {'name': 'log.info', 'changes': {'ret': True}, 'comment': 'Module function log.info executed', 'result': True, '__sls__': 'states.scratch.first', '__run_num__': 0, 'start_time': '22:32:13.493369', 'duration': 0.0, '__id__': 'log first'}, 'cmd_|-install vs build tools 1_|-Start-Process -Wait -FilePath "C:\\windows\\TEMP\\vs_BuildTools.exe" -ArgumentList "--passive"_|-run': {'name': 'Start-Process -Wait -FilePath "C:\\windows\\TEMP\\vs_BuildTools.exe" -ArgumentList "--passive"', 'changes': {'pid': 3808, 'retcode': 0, 'stdout': '', 'stderr': ''}, 'result': True, 'comment': 'Command "Start-Process -Wait -FilePath "C:\\windows\\TEMP\\vs_BuildTools.exe" -ArgumentList "--passive"" run', '__sls__': 'states.scratch.first', '__run_num__': 1, 'start_time': '22:32:13.507003', 'duration': 47085.106, '__id__': 'install vs build tools 1'}}, 'retcode': 0, 'jid': '20201015223200590679', 'fun': 'state.sls', 'fun_args': ['states.scratch.first', {'queue': False, 'concurrent': False}]}

Master output:

[DEBUG   ] Sending event: tag = salt/run/20201015223134158159/ret; data = {'fun': 'runner.state.orch', 'jid': '20201015223134158159', 'user': 'sudo_azureuser', 'fun_args': ['orch.scratch', {'orchestration_jid': '20201015223134158159'}], '_stamp': '2020-10-15T22:32:15.862410', 'return': {'data': {'salt-master_master': {'salt_|-Do state_|-Do state_|-state': {'name': 'Do state', 'changes': {'out': 'highstate', 'ret': {'salt-minion-new': False}}, 'comment': 'Run failed on minions: salt-minion-new', 'result': False, '__sls__': 'orch.scratch', '__run_num__': 0, 'start_time': '22:31:35.101771', 'duration': 25431.29, '__id__': 'Do state'}, 'salt_|-Do state again_|-Do state again_|-state': {'name': 'Do state again', 'changes': {'out': 'highstate', 'ret': {'salt-minion-new': False}}, 'comment': 'Run failed on minions: salt-minion-new', 'result': False, '__sls__': 'orch.scratch', '__run_num__': 1, 'start_time': '22:32:00.533627', 'duration': 15325.259, '__id__': 'Do state again'}}}, 'outputter': 'highstate', 'retcode': 1}, 'success': False}
[DEBUG   ] LazyLoaded highstate.output
salt-master_master:
----------
          ID: Do state
    Function: salt.state
      Result: False
     Comment: Run failed on minions: salt-minion-new
     Started: 22:31:35.101771
    Duration: 25431.29 ms
     Changes:
              salt-minion-new:
                  False
----------
          ID: Do state again
    Function: salt.state
      Result: False
     Comment: Run failed on minions: salt-minion-new
     Started: 22:32:00.533627
    Duration: 15325.259 ms
     Changes:
              salt-minion-new:
                  False

Summary for salt-master_master
------------
Succeeded: 0 (changed=2)
Failed:    2
------------
Total states run:     2
Total run time:  40.757 s

Setup
3 files: scratch.sls, first.sls, vs_BuildTools.exe
Clean VM running Windows 2019

salt://orch/scratch:

Do state:
  salt.state:
    - sls: states.scratch.first
    - tgt: 'salt-minion-new'
Do state again:
  salt.state:
    - sls: states.scratch.first
    - tgt: 'salt-minion-new'

salt://states/scratch/first:

log first:
  module.run:
    - name: log.info
    - message: Starting first
install vs build tools 1:
  cmd.run:
    - name: Start-Process -Wait -FilePath "{{ salt['environ.get']('temp') }}\vs_BuildTools.exe" -ArgumentList "--passive"
    - shell: powershell

vs_BuildTools.exe from here: https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2019

Steps to Reproduce the behavior
This works ✔️: salt -t 60 'salt-minion-new' state.apply states.scratch.first -l debug
This fails ❌: salt-run -t 60 state.orch orch.scratch -l debug

Expected behavior
salt-run state.orch orch.scratch should succeed

Screenshots
image

Versions Report

salt --versions-report on master (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
           Salt: 3001.1
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: 2.0.3
      gitpython: 2.1.8
         Jinja2: 2.10
        libgit2: 0.26.0
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: 3.4.7
         pygit2: 0.26.2
         Python: 3.6.9 (default, Jul 17 2020, 12:50:27)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 17.1.2
          smmap: 2.0.3
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: ubuntu 18.04 Bionic Beaver
         locale: UTF-8
        machine: x86_64
        release: 5.4.0-1031-azure
         system: Linux
        version: Ubuntu 18.04 Bionic Beaver
Minion: `Salt-Minion-3001.1-Py3-AMD64-Setup.exe`

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.