saltstack / saltstack/salt

[BUG] 3006: salt-api OverflowError: Integer value out of range when trying to get the async job status using pepper client

Open
#67,957 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
The issue happens when I am trying to get the status or description of the salt job using the pepper client. it seems there is a bug in salt-api/msgpack in the serialization/deserialization procedure between salt-api and salt-master.

salt-api[788540]: [TRACE   ] ReqChannel send clear load={'fun': 'jobs.print_job', 'kwarg': {'kwarg': {}, 'arg': [20250410151117305438]}, 'token': 'ffecc2e82ca9b4d879479c3e97a33aa27d721c337631e7dc66c2f55d28fa2df3', 'client': 'runner', 'cmd': 'runner'}
salt-api[788540]: [TRACE   ] Failed to send msg OverflowError('Integer value out of range')
salt-api[788540]: [DEBUG   ] Closing AsyncReqChannel instance
salt-api[788540]: [DEBUG   ] Closing IPCMessageSubscriber instance
salt-api[788540]: [DEBUG   ] Error while processing request for: /
salt-api[788540]: Traceback (most recent call last):
salt-api[788540]:   File "msgpack/_packer.pyx", line 171, in msgpack._cmsgpack.Packer._pack_inner
salt-api[788540]: OverflowError: Python int too large to convert to C unsigned long
salt-api[788540]: During handling of the above exception, another exception occurred:
salt-api[788540]: Traceback (most recent call last):
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/netapi/rest_cherrypy/app.py", line 860, in hypermedia_handler
salt-api[788540]:     ret = cherrypy.serving.request._hypermedia_inner_handler(*args, **kwargs)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
salt-api[788540]:     return self.callable(*self.args, **self.kwargs)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/netapi/rest_cherrypy/app.py", line 1330, in POST
salt-api[788540]:     return {"return": list(self.exec_lowstate(token=cherrypy.session.get("token")))}
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/netapi/rest_cherrypy/app.py", line 1227, in exec_lowstate
salt-api[788540]:     ret = self.api.run(chunk)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/netapi/__init__.py", line 181, in run
salt-api[788540]:     return l_fun(*f_call.get("args", ()), **f_call.get("kwargs", {}))
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/netapi/__init__.py", line 262, in runner
salt-api[788540]:     return runner.cmd_sync(kwargs, timeout=timeout, full_return=full_return)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/runner.py", line 145, in cmd_sync
salt-api[788540]:     return mixins.SyncClientMixin.cmd_sync(
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/mixins.py", line 190, in cmd_sync
salt-api[788540]:     job = self.master_call(**low)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/mixins.py", line 165, in master_call
salt-api[788540]:     ret = channel.send(load)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 125, in wrap
salt-api[788540]:     raise exc_info[1].with_traceback(exc_info[2])
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 131, in _target
salt-api[788540]:     result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/ioloop.py", line 459, in run_sync
salt-api[788540]:     return future_cell[0].result()
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/concurrent.py", line 249, in result
salt-api[788540]:     raise_exc_info(self._exc_info)
salt-api[788540]:   File "<string>", line 4, in raise_exc_info
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 1064, in run
salt-api[788540]:     yielded = self.gen.throw(*exc_info)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/channel/client.py", line 324, in send
salt-api[788540]:     ret = yield self._uncrypted_transfer(load, timeout=timeout)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 1056, in run
salt-api[788540]:     value = future.result()
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/concurrent.py", line 249, in result
salt-api[788540]:     raise_exc_info(self._exc_info)
salt-api[788540]:   File "<string>", line 4, in raise_exc_info
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 1064, in run
salt-api[788540]:     yielded = self.gen.throw(*exc_info)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/channel/client.py", line 295, in _uncrypted_transfer
salt-api[788540]:     ret = yield self.transport.send(
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 1056, in run
salt-api[788540]:     value = future.result()
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/concurrent.py", line 249, in result
salt-api[788540]:     raise_exc_info(self._exc_info)
salt-api[788540]:   File "<string>", line 4, in raise_exc_info
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 1064, in run
salt-api[788540]:     yielded = self.gen.throw(*exc_info)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/transport/tcp.py", line 1062, in send
salt-api[788540]:     ret = yield self.message_client.send(load, timeout=timeout)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 1056, in run
salt-api[788540]:     value = future.result()
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/concurrent.py", line 249, in result
salt-api[788540]:     raise_exc_info(self._exc_info)
salt-api[788540]:   File "<string>", line 4, in raise_exc_info
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 309, in wrapper
salt-api[788540]:     yielded = next(result)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/transport/tcp.py", line 761, in send
salt-api[788540]:     item = salt.transport.frame.frame_msg(msg, header=header)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/transport/frame.py", line 18, in frame_msg
salt-api[788540]:     return salt.utils.msgpack.dumps(framed_msg)
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/msgpack.py", line 113, in packb
salt-api[788540]:     return msgpack.packb(o, **_sanitize_msgpack_kwargs(kwargs))
salt-api[788540]:   File "/opt/saltstack/salt/lib/python3.10/site-packages/msgpack/__init__.py", line 36, in packb
salt-api[788540]:     return Packer(**kwargs).pack(o)
salt-api[788540]:   File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer.pack
salt-api[788540]:   File "msgpack/_packer.pyx", line 276, in msgpack._cmsgpack.Packer.pack
salt-api[788540]:   File "msgpack/_packer.pyx", line 270, in msgpack._cmsgpack.Packer._pack
salt-api[788540]:   File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
salt-api[788540]:   File "msgpack/_packer.pyx", line 270, in msgpack._cmsgpack.Packer._pack
salt-api[788540]:   File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
salt-api[788540]:   File "msgpack/_packer.pyx", line 270, in msgpack._cmsgpack.Packer._pack
salt-api[788540]:   File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
salt-api[788540]:   File "msgpack/_packer.pyx", line 270, in msgpack._cmsgpack.Packer._pack
salt-api[788540]:   File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
salt-api[788540]:   File "msgpack/_packer.pyx", line 270, in msgpack._cmsgpack.Packer._pack
salt-api[788540]:   File "msgpack/_packer.pyx", line 232, in msgpack._cmsgpack.Packer._pack_inner
salt-api[788540]:   File "msgpack/_packer.pyx", line 270, in msgpack._cmsgpack.Packer._pack
salt-api[788540]:   File "msgpack/_packer.pyx", line 180, in msgpack._cmsgpack.Packer._pack_inner
salt-api[788540]: OverflowError: Integer value out of range
salt-api[788540]: 192.168.88.0 - - [10/Apr/2025:16:30:19] "POST / HTTP/1.1" 500 6360 "" "Python-urllib/3.10"

Setup
I used pip3 to install salt-master, salt-api and salt-minions

Please be as specific as possible and give set-up details.

  • [ x] VM running on a cloud service, please be explicit and add details
  • [ x] used pip to bootstrap and install

Steps to Reproduce the behavior
pepper -vvvv --client=runner jobs.print_job '20250410151117305438' missing=True

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with salt/transport/frame.py and salt/utils/msgpack.py, following the pepper jobs.print_job reproduction and the traceback through salt-api. Investigate how the large job ID is serialized between salt-api and salt-master, then verify that requesting the reported job status no longer raises OverflowError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.