grains_refresh_every directive not reliably creating schedule
@xeacott is already working on this.
Since Feb 7, 2020.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue
Setting grains_refresh_every does not reliably create a schedule in _schedule.conf. Running the minion in debug shows that the directive is processed, but immediately appears to get deleted.
Relevant debug log
C:\salt>salt-minion-debug
The salt-minion service is not started.
More help is available by typing NET HELPMSG 3521.
[DEBUG ] Reading configuration from c:\salt\conf\minion
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d_schedule.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d_schedule.conf
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d\network.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d\network.conf
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d\rainhail_minion_win.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d\rainhail_minion_win.conf
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d\s3.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d\s3.conf
[DEBUG ] Multiprocessing queue logging configured for the process running under PID: 8492 at log level 10
[DEBUG ] Marking 'is_ip' as a jinja filter
[DEBUG ] Marking 'is_ipv4' as a jinja filter
[DEBUG ] Marking 'is_ipv6' as a jinja filter
[DEBUG ] Marking 'ipv4' as a jinja filter
[DEBUG ] Marking 'ipv6' as a jinja filter
[DEBUG ] Marking 'ipaddr' as a jinja filter
[DEBUG ] Marking 'ip_host' as a jinja filter
[DEBUG ] Marking 'network_hosts' as a jinja filter
[DEBUG ] Marking 'network_size' as a jinja filter
[DEBUG ] Marking 'gen_mac' as a jinja filter
[DEBUG ] Marking 'mac_str_to_bytes' as a jinja filter
[DEBUG ] Marking 'dns_check' as a jinja filter
[DEBUG ] Multiprocessing queue logging configured for the process running under PID: 8492 at log level 10
[DEBUG ] Configuration file path: c:\salt\conf\minion
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO ] Setting up the Salt Minion "demotom038.rainhail.com"
[DEBUG ] Marking 'base64_encode' as a jinja filter
[DEBUG ] Marking 'base64_decode' as a jinja filter
[DEBUG ] Marking 'md5' as a jinja filter
[DEBUG ] Marking 'sha256' as a jinja filter
[DEBUG ] Marking 'sha512' as a jinja filter
[DEBUG ] Marking 'hmac' as a jinja filter
[DEBUG ] Marking 'random_hash' as a jinja filter
[DEBUG ] Marking 'rand_str' as a jinja filter
[DEBUG ] Marking 'file_hashsum' as a jinja filter
[DEBUG ] Marking 'http_query' as a jinja filter
[DEBUG ] Marking 'strftime' as a jinja filter
[DEBUG ] Marking 'date_format' as a jinja filter
[DEBUG ] Marking 'yaml_dquote' as a jinja filter
[DEBUG ] Marking 'yaml_squote' as a jinja filter
[DEBUG ] Marking 'yaml_encode' as a jinja filter
[DEBUG ] Marking 'raise' as a jinja global
[DEBUG ] Marking 'match' as a jinja test
[DEBUG ] Marking 'equalto' as a jinja test
[DEBUG ] Marking 'skip' as a jinja filter
[DEBUG ] Marking 'sequence' as a jinja filter
[DEBUG ] Marking 'to_bool' as a jinja filter
[DEBUG ] Marking 'tojson' as a jinja filter
[DEBUG ] Marking 'quote' as a jinja filter
[DEBUG ] Marking 'regex_escape' as a jinja filter
[DEBUG ] Marking 'regex_search' as a jinja filter
[DEBUG ] Marking 'regex_match' as a jinja filter
[DEBUG ] Marking 'regex_replace' as a jinja filter
[DEBUG ] Marking 'uuid' as a jinja filter
[DEBUG ] Marking 'unique' as a jinja filter
[DEBUG ] Marking 'min' as a jinja filter
[DEBUG ] Marking 'max' as a jinja filter
[DEBUG ] Marking 'avg' as a jinja filter
[DEBUG ] Marking 'union' as a jinja filter
[DEBUG ] Marking 'intersect' as a jinja filter
[DEBUG ] Marking 'difference' as a jinja filter
[DEBUG ] Marking 'symmetric_difference' as a jinja filter
[DEBUG ] Created pidfile: c:\salt\var\run\salt-minion.pid
[INFO ] Starting up the Salt Minion
[DEBUG ] AsyncEventPublisher PUB socket URI: 4510
[DEBUG ] AsyncEventPublisher PULL socket URI: 4511
[INFO ] Starting pull socket on 4511
[DEBUG ] SaltEvent PUB socket URI: 4510
[DEBUG ] SaltEvent PULL socket URI: 4511
[DEBUG ] Initializing new IPCClient for path: 4510
[DEBUG ] Grains cache last modified 406 seconds ago and cache expiration is set to 300. Grains cache expired. Refreshing.
[DEBUG ] Reading configuration from c:\salt\conf\minion
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d_schedule.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d_schedule.conf
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d\network.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d\network.conf
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d\rainhail_minion_win.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d\rainhail_minion_win.conf
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d\s3.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d\s3.conf
[INFO ] Retrieved install_date from file
[DEBUG ] Reading configuration from c:\salt\conf\minion
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d_schedule.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d_schedule.conf
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d\network.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d\network.conf
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d\rainhail_minion_win.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d\rainhail_minion_win.conf
[DEBUG ] Including configuration from 'c:\salt\conf\minion.d\s3.conf'
[DEBUG ] Reading configuration from c:\salt\conf\minion.d\s3.conf
[DEBUG ] Requesting URL http://webadmin.rainhail.com/public/all_tomcat_hosts.php using POST method
[DEBUG ] Using backend: tornado
[DEBUG ] Response Status Code: 200
[INFO ] Creating minion process manager
[DEBUG ] Process Manager starting!
[DEBUG ] Process Manager starting!
[DEBUG ] Connecting to master. Attempt 1 (infinite attempts)
[DEBUG ] Master URI: tcp://172.24.55.45:4506
[DEBUG ] Popen(['git', 'version'], cwd=C:\salt, universal_newlines=False, shell=None)
[DEBUG ] Initializing new AsyncAuth for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506')
[DEBUG ] Generated random reconnect delay between '1000ms' and '11000ms' (4292)
[DEBUG ] Setting zmq_reconnect_ivl to '4292ms'
[DEBUG ] Setting zmq_reconnect_ivl_max to '11000ms'
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506', 'clear')
[DEBUG ] Connecting the Minion to the Master URI (for the return server): tcp://172.24.55.45:4506
[DEBUG ] Trying to connect to: tcp://172.24.55.45:4506
[DEBUG ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG ] Decrypting the current master AES key
[DEBUG ] salt.crypt.get_rsa_key: Loading private key
[DEBUG ] salt.crypt._get_key_with_evict: Loading private key
[DEBUG ] Loaded minion key: c:\salt\conf\pki\minion\minion.pem
[DEBUG ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG ] Closing AsyncZeroMQReqChannel instance
[DEBUG ] Connecting the Minion to the Master publish port, using the URI: tcp://172.24.55.45:4505
[DEBUG ] salt.crypt.get_rsa_key: Loading private key
[DEBUG ] Loaded minion key: c:\salt\conf\pki\minion\minion.pem
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506', 'aes')
[DEBUG ] Re-using AsyncAuth for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506')
[DEBUG ] Connecting the Minion to the Master URI (for the return server): tcp://172.24.55.45:4506
[DEBUG ] Trying to connect to: tcp://172.24.55.45:4506
[DEBUG ] salt.crypt.get_rsa_key: Loading private key
[DEBUG ] Loaded minion key: c:\salt\conf\pki\minion\minion.pem
[DEBUG ] Closing AsyncZeroMQReqChannel instance
[DEBUG ] Retrieving grains from cache
[DEBUG ] Initializing new Schedule
[DEBUG ] LazyLoaded platform.is_windows
[DEBUG ] LazyLoaded path.which
[DEBUG ] LazyLoaded timezone.get_offset
[DEBUG ] LazyLoaded cmd.run_all
[INFO ] Executing command ['tzutil', '/g'] in directory 'C:\Users\heiselmanjx'
[DEBUG ] stdout: Central Standard Time
[DEBUG ] LazyLoaded config.merge
[DEBUG ] SaltEvent PUB socket URI: 4510
[DEBUG ] SaltEvent PULL socket URI: 4511
[DEBUG ] Initializing new IPCClient for path: 4511
[DEBUG ] Sending event: tag = /salt/minion/minion_schedule_delete_complete; data = {'schedule': {'__mine_interval': {'maxrunning': 2, 'function': 'mine.update', 'minutes': 60, 'jid_include': True, 'enabled': True, 'return_job': False, 'run_on_start': True}}, 'complete': True, '_stamp': '2020-01-28T14:34:51.763513'}
[DEBUG ] Persisting schedule
[DEBUG ] Closing IPCMessageClient instance
[DEBUG ] LazyLoaded mine.update
[INFO ] Updating job settings for scheduled job: __mine_interval
[DEBUG ] SaltEvent PUB socket URI: 4510
[DEBUG ] SaltEvent PULL socket URI: 4511
[DEBUG ] Initializing new IPCClient for path: 4511
[DEBUG ] Sending event: tag = /salt/minion/minion_schedule_add_complete; data = {'schedule': {'__mine_interval': {'maxrunning': 2, 'function': 'mine.update', 'enabled': True, 'jid_include': True, 'minutes': 60, 'return_job': False, 'run_on_start': True}}, 'complete': True, '_stamp': '2020-01-28T14:34:51.763513'}
[DEBUG ] Persisting schedule
[DEBUG ] Closing IPCMessageClient instance
[INFO ] Added mine.update to scheduler
[DEBUG ] SaltEvent PUB socket URI: 4510
[DEBUG ] SaltEvent PULL socket URI: 4511
[DEBUG ] Initializing new IPCClient for path: 4511
[DEBUG ] Sending event: tag = /salt/minion/minion_schedule_delete_complete; data = {'schedule': {'__mine_interval': {'maxrunning': 2, 'function': 'mine.update', 'enabled': True, 'jid_include': True, 'minutes': 60, 'return_job': False, 'run_on_start': True}}, 'complete': True, '_stamp': '2020-01-28T14:34:51.763513'}
[DEBUG ] Persisting schedule
[DEBUG ] Closing IPCMessageClient instance
[DEBUG ] SaltEvent PUB socket URI: 4510
[DEBUG ] SaltEvent PULL socket URI: 4511
[DEBUG ] Initializing new IPCClient for path: 4511
[DEBUG ] Sending event: tag = /salt/minion/minion_schedule_delete_complete; data = {'schedule': {'__mine_interval': {'maxrunning': 2, 'function': 'mine.update', 'enabled': True, 'jid_include': True, 'minutes': 60, 'return_job': False, 'run_on_start': True}}, 'complete': True, '_stamp': '2020-01-28T14:34:51.763513'}
[DEBUG ] Persisting schedule
[DEBUG ] Closing IPCMessageClient instance
[INFO ] Minion is starting as user 'RAINHAIL\heiselmanjx'
[DEBUG ] Minion 'demotom038.rainhail.com' trying to tune in
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506', 'aes')
[DEBUG ] Initializing new AsyncAuth for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506')
[DEBUG ] Connecting the Minion to the Master URI (for the return server): tcp://172.24.55.45:4506
[DEBUG ] Trying to connect to: tcp://172.24.55.45:4506
[DEBUG ] Closing AsyncZeroMQReqChannel instance
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506', 'aes')
[DEBUG ] Initializing new AsyncAuth for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506')
[DEBUG ] Connecting the Minion to the Master URI (for the return server): tcp://172.24.55.45:4506
[DEBUG ] Trying to connect to: tcp://172.24.55.45:4506
[DEBUG ] Closing AsyncZeroMQReqChannel instance
[INFO ] Minion is ready to receive requests!
[DEBUG ] Enabling the grains refresher. Will run every 720 minutes.
[DEBUG ] Minion of 'webadminubuntu02.rainhail.com' is handling event tag '/salt/minion/minion_schedule_delete_complete'
[DEBUG ] Minion of 'webadminubuntu02.rainhail.com' is handling event tag '/salt/minion/minion_schedule_add_complete'
[DEBUG ] Minion of 'webadminubuntu02.rainhail.com' is handling event tag '/salt/minion/minion_schedule_delete_complete'
[DEBUG ] Minion of 'webadminubuntu02.rainhail.com' is handling event tag '/salt/minion/minion_schedule_delete_complete'
[DEBUG ] schedule: Job __mine_interval was scheduled with jid_include, adding to cache (jid_include defaults to True)
[DEBUG ] schedule: Job __mine_interval was scheduled with a max number of 2
[INFO ] Running scheduled job: __mine_interval
[DEBUG ] Multiprocessing queue logging configured for the process running under PID: 6000 at log level 10
[DEBUG ] schedule.handle_func: adding this job to the jobcache with data {'schedule': '__mine_interval', 'fun_args': [], 'fun': 'mine.update', 'pid': 6000, 'id': 'demotom038.rainhail.com', 'jid': '20200128083454372907'}
[DEBUG ] LazyLoaded mine.update
[DEBUG ] LazyLoaded config.merge
[DEBUG ] schedule.handle_func: Removing c:\salt\var\cache\salt\minion\proc\20200128083454372907
[DEBUG ] Popen(['git', 'version'], cwd=C:\salt, universal_newlines=False, shell=None)
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506', 'aes')
[DEBUG ] Initializing new AsyncAuth for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506')
[DEBUG ] Connecting the Minion to the Master URI (for the return server): tcp://172.24.55.45:4506
[DEBUG ] Trying to connect to: tcp://172.24.55.45:4506
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('c:\salt\conf\pki\minion', 'demotom038.rainhail.com', 'tcp://172.24.55.45:4506', 'clear')
[DEBUG ] Connecting the Minion to the Master URI (for the return server): tcp://172.24.55.45:4506
[DEBUG ] Trying to connect to: tcp://172.24.55.45:4506
[DEBUG ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG ] Decrypting the current master AES key
[DEBUG ] salt.crypt.get_rsa_key: Loading private key
[DEBUG ] salt.crypt._get_key_with_evict: Loading private key
[DEBUG ] Loaded minion key: c:\salt\conf\pki\minion\minion.pem
[DEBUG ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG ] Closing AsyncZeroMQReqChannel instance
[DEBUG ] Closing AsyncZeroMQReqChannel instance
The Salt Minion is shutdown. Minion received a SIGINT. Exited.
Terminate batch job (Y/N)? y
Setup
In /etc/salt/minion or C:\salt\conf\minion:
grains_refresh_every: 720
Steps to Reproduce Issue
Observe that the _schedule.conf file doesn't get updated with the __update_grains schedule item
Versions Report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: 1.11.5
cherrypy: 17.4.1
dateutil: 2.7.5
docker-py: Not Installed
gitdb: 2.0.5
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: 1.6.1
M2Crypto: Not Installed
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: 2.19
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)]
python-gnupg: 0.4.3
PyYAML: 3.13
PyZMQ: 17.1.2
RAET: Not Installed
smmap: 2.0.5
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.2.5
System Versions:
dist:
locale: cp1252
machine: AMD64
release: 2019Server
system: Windows
version: 2019Server 10.0.17763 SP0 Multiprocessor Free
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.