saltstack / saltstack/salt

[BUG] serialization issue in kubernetes show_namespace through salt-api

Open
#61,371 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
When the kubernetes.show_namespace function is called through the salt-api it returns an "500 Internal Server Error".
Calling the same function through the salt commandline does work.

root@salt-master-55d7dc7477-vwdmz:/# curl -i --insecure -X POST https://localhost:8001 -H 'Content-type: application/json' -H 'X-Auth-Token:0c99cf51e4eb713b690e44ea08f3d8271cc89583' -d '{"client":"local","tgt":"8YVR363","fun":"kubernetes.show_namespace","arg":["test123"]}'
HTTP/1.1 500 Internal Server Error
Content-Type: text/html;charset=utf-8
Server: CherryPy/18.6.1
Date: Wed, 15 Dec 2021 14:19:16 GMT
Allow: GET, HEAD, POST
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: GET, POST
Access-Control-Allow-Credentials: true
Cache-Control: private
Content-Length: 2324
Vary: Accept-Encoding
Set-Cookie: session_id=0c99cf51e4eb713b690e44ea08f3d8271cc89583; expires=Thu, 16 Dec 2021 00:19:16 GMT; Max-Age=36000; Path=/

<!DOCTYPE html PUBLIC
"-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta>
    <title>500 Internal Server Error</title>
    <style type="text/css">
    #powered_by {
        margin-top: 20px;
        border-top: 2px solid black;
        font-style: italic;
    }

    #traceback {
        color: red;
    }
    </style>
</head>
    <body>
        <h2>500 Internal Server Error</h2>
        <p>Could not serialize the return data from Salt.</p>
        <pre id="traceback">Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/netapi/rest_cherrypy/app.py", line 909, in hypermedia_handler
    response = out(ret)
  File "/usr/lib/python3/dist-packages/salt/utils/json.py", line 149, in dumps
    return json_module.dumps(obj, **kwargs)  # future lint: blacklisted-function
  File "/usr/lib/python3.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type datetime is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/cherrypy/_cprequest.py", line 638, in respond
    self._do_respond(path_info)
  File "/usr/local/lib/python3.7/dist-packages/cherrypy/_cprequest.py", line 697, in _do_respond
    response.body = self.handler()
  File "/usr/local/lib/python3.7/dist-packages/cherrypy/lib/encoding.py", line 223, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/netapi/rest_cherrypy/app.py", line 914, in hypermedia_handler
    raise cherrypy.HTTPError(500, msg)
cherrypy._cperror.HTTPError: (500, 'Could not serialize the return data from Salt.')
</pre>
    <div id="powered_by">
      <span>
        Powered by <a href="http://www.cherrypy.org">CherryPy 18.6.1</a>
      </span>
    </div>
    </body>
</html>
root@salt-master-55d7dc7477-vwdmz:/# salt 8YVR363 kubernetes.show_namespace test123
8YVR363:
    ----------
    api_version:
        v1
    kind:
        Namespace
    metadata:
        ----------
        annotations:
            None
        cluster_name:
            None
        creation_timestamp:
        deletion_grace_period_seconds:
            None
        deletion_timestamp:
            None
        finalizers:
            None
        generate_name:
            None
        generation:
            None
        labels:
            None
        managed_fields:
            |_
              ----------
              api_version:
                  v1
              fields_type:
                  FieldsV1
              fields_v1:
                  ----------
                  f:status:
                      ----------
                      f:phase:
                          ----------
              manager:
                  OpenAPI-Generator
              operation:
                  Update
              time:
        name:
            test123
        namespace:
            None
        owner_references:
            None
        resource_version:
            101416
        self_link:
            None
        uid:
            72e1a2b1-0afe-426a-bde7-ef516e354ec6
    spec:
        ----------
        finalizers:
            - kubernetes
    status:
        ----------
        conditions:
            None
        phase:
            Active
root@salt-master-55d7dc7477-vwdmz:/# salt 8YVR363 kubernetes.show_namespace test123 --out raw
{'8YVR363': {'api_version': 'v1', 'kind': 'Namespace', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2021, 12, 15, 12, 48, 40), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields_type': 'FieldsV1', 'fields_v1': {'f:status': {'f:phase': {}}}, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2021, 12, 15, 12, 48, 40)}], 'name': 'test123', 'namespace': None, 'owner_references': None, 'resource_version': '101416', 'self_link': None, 'uid': '72e1a2b1-0afe-426a-bde7-ef516e354ec6'}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'conditions': None, 'phase': 'Active'}}}

In the raw output from the commandline it can be observed that a 'creation_timestamp': datetime.datetime(2021, 12, 15, 12, 48, 40), object is returned, which likely corresponds with the error of the server TypeError: Object of type datetime is not JSON serializable.

Setup

Our salt-minion is running on-prem directly on the kubernetes host.
Our salt-master is running in a container, together with the salt-api in a different kubernetes cluster.
The salt-minion connects to the service IP of the salt-master container.

  • on-prem machine
  • VM (Virtualbox, KVM, etc. please specify)
  • VM running on a cloud service, please be explicit and add details
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD

Steps to Reproduce the behavior
Call the salt-api with:
curl -i --insecure -X POST https://localhost:8001 -H 'Content-type: application/json' -H 'X-Auth-Token:<auth-token>' -d '{"client":"local","tgt":"<minion-id>","fun":"kubernetes.show_namespace","arg":["<namespace-to-show>"]}'
on a salt-minion running on a kubernetes host and query for a specific, existing namespace.

Expected behavior
The return should be similar to the cmdline output:

root@salt-master-55d7dc7477-vwdmz:/# salt 8YVR363 kubernetes.show_namespace test123 --out json
{
    "8YVR363": {
        "api_version": "v1",
        "kind": "Namespace",
        "metadata": {
            "annotations": null,
            "cluster_name": null,
            "creation_timestamp": "datetime.datetime(2021, 12, 15, 12, 48, 40)",
            "deletion_grace_period_seconds": null,
            "deletion_timestamp": null,
            "finalizers": null,
            "generate_name": null,
            "generation": null,
            "labels": null,
            "managed_fields": [
                {
                    "api_version": "v1",
                    "fields_type": "FieldsV1",
                    "fields_v1": {
                        "f:status": {
                            "f:phase": {}
                        }
                    },
                    "manager": "OpenAPI-Generator",
                    "operation": "Update",
                    "time": "datetime.datetime(2021, 12, 15, 12, 48, 40)"
                }
            ],
            "name": "test123",
            "namespace": null,
            "owner_references": null,
            "resource_version": "101416",
            "self_link": null,
            "uid": "72e1a2b1-0afe-426a-bde7-ef516e354ec6"
        },
        "spec": {
            "finalizers": [
                "kubernetes"
            ]
        },
        "status": {
            "conditions": null,
            "phase": "Active"
        }
    }
}

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) 
root@salt-master-55d7dc7477-vwdmz:/# salt-run --versions-report
Salt Version:
          Salt: 3003.3

Dependency Versions:
          cffi: Not Installed
      cherrypy: 18.6.1
      dateutil: 2.7.3
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10
       libgit2: 0.27.7
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 0.5.6
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.19
      pycrypto: Not Installed
  pycryptodome: 3.6.1
        pygit2: 0.27.4
        Python: 3.7.3 (default, Jan 22 2021, 20:04:44)
  python-gnupg: Not Installed
        PyYAML: 3.13
         PyZMQ: 17.1.2
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.1

System Versions:
          dist: debian 10 buster
        locale: utf-8
       machine: x86_64
       release: 5.4.109+
        system: Linux
       version: Debian GNU/Linux 10 buster

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in salt/netapi/rest_cherrypy/app.py at hypermedia_handler and inspect the serialization call in salt/utils/json.py. Reproduce the issue with the provided salt-api curl request and compare it with the command-line output. Done means kubernetes.show_namespace returns JSON through salt-api instead of a 500 serialization error.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python
Domain
api, devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.