saltstack / saltstack/salt

[BUG] Minion throws exception with timezone offset aware 'when' property in the 'schedule' module

Open
#61,432 2 comments 1 reaction 1 assignee View on GitHub

@krionbsd is already working on this.

Since Jan 8, 2022.

bug severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
The 'schedule' module contains a 'when' property for which it can accept the format supported by dateutil parser. On observation, when the datetime string contains the timezone then the master daemon accepts this message and publishes an event on the zeromq but the minion doesn't accept it instead throws an exception on evaluation of the schedule item.

2022-01-07 22:31:25,282 [salt.minion      :478 ][ERROR   ][25373] Exception can't compare offset-naive and offset-aware datetimes occurred in scheduled job

Setup

  • on-prem machine
  • VM (Virtualbox, KVM, etc. please specify)
  • VM running on a cloud service, please be explicit and add details
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD

Steps to Reproduce the behavior

  • Add the schedule on minion with timezone offset on syndic master.
root@salt-syndic-2:~# salt testminon schedule.add j1 function='test.ping' when='2022-01-07T03:14:39+05:30'
testminon:
    ----------
    comment:
        Added job: j1 to schedule.
    result:
        True
  • List the schedule is successfully published on zeromq.
      j1:
        enabled: true
        function: test.ping
        jid_include: true
        maxrunning: 1
        name: j1
        when: '2022-01-07T03:14:39+05:30'
  • On minion, the schedule is added to _schedule.conf (/etc/salt/minion.d/_schedule.conf)
  j1: {enabled: true, function: test.ping, jid_include: true, maxrunning: 1, name: j1,
    when: '2022-01-07T03:14:39+05:30'}
  • On minion, in logs /var/log/salt/minion we see an exception almost immediately when it tries to load the schedule 'j1' and with infinite log exception lines printed every second.
2022-01-07 22:43:29,495 [salt.minion      :1799][DEBUG   ][92685] minion return: {'success': True, 'fun_args': ['j1', {'function': 'test.ping', 'when': '2022-01-07T03:14:39+05:30'}], 'jid': '20220108064329379309', 'retcode': 0, 'fun': 'schedule.add', 'return': {'comment': 'Added job: j1 to schedule.', 'result': True}, 'master_id': 'salt-syndic-2'}
2022-01-07 22:43:30,430 [salt.utils.schedule:39  ][TRACE   ][92599] ==== evaluating schedule now None =====
2022-01-07 22:43:32,281 [salt.minion      :478 ][ERROR   ][25373] Exception can't compare offset-naive and offset-aware datetimes occurred in scheduled job
2022-01-07 22:43:33,282 [salt.minion      :478 ][ERROR   ][25373] Exception can't compare offset-naive and offset-aware datetimes occurred in scheduled job

Expected behavior
We expect to see the schedule is loaded on the minion and follows the timezone offset as the documentation points that schedular 'when' property supports the dateutil format (ref: https://docs.saltproject.io/en/latest/ref/states/all/salt.states.schedule.html)

Screenshots
None

Versions Report

salt --versions-report
Salt Version:
           Salt: 3000.5

Dependency Versions:
           cffi: Not Installed
       cherrypy: 3.5.0
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: 2.0.0
      gitpython: 2.1.1
         Jinja2: 2.9.4
        libgit2: Not Installed
       M2Crypto: 0.21.1
           Mako: 1.0.0
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.9 (default, Jun 21 2019, 00:38:53)
   python-gnupg: 0.3.6
         PyYAML: 3.11
          PyZMQ: 14.4.0
          smmap: 2.0.1
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.0.5

System Versions:
           dist: debian 8.4
         locale: ANSI_X3.4-1968
        machine: x86_64
        release: 3.16.0-4-amd64
         system: Linux
        version: debian 8.4

Additional context
There is a comparison in the eval() function between now and the scheduled datetime, the exception is thrown as now is not instantiated with timezone offset awareness.
for ref:
[evaluator] now() [not timezone aware]: https://github.com/saltstack/salt/blob/master/salt/utils/schedule.py#L1510
[evaluator] eval() [compare with schedule time]: https://github.com/saltstack/salt/blob/master/salt/utils/schedule.py#L1111
[minion-scheduler] exception-trace: https://github.com/saltstack/salt/blob/master/salt/minion.py#L503

A probable fix could be - to make now() & the schedule time, timezone aware. If they are provided with timezone offset, then follow it otherwise patch the timezone with the local tz.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.