saltstack / saltstack/salt

[BUG] localfs cache not working properly with rest_cherrypy api

Open
#62,180 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage Salt-API
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
I am trying to cache grains of a minion and read it over HTTP API. It works as expected when I try with CLI command, but I kepp getting same error with HTTP API.

Setup
We run salt master in a docker container with following config:

/ # more /etc/salt/master.d/api.conf
rest_cherrypy:
  disable_ssl: True
  webhook_url: /hook
  host: 127.0.0.1
  port: 8000
  webhook_disable_auth: True
/ # more /etc/salt/master.d/master.conf
cachedir: "/var/cache/salt"
keep_jobs: 3
gather_job_timeout: 30
timeout: 30
con_cache: true
external_auth:
  pam:
    root:
      - '.*'
      - '@runner'
      - '@wheel'
  sharedsecret:
    saltapi:
      - '.*'
      - '@runner'
      - '@wheel'
      - '@jobs'

sharedsecret: ***

state_top: "top.sls"
state_verbose: false

file_roots:
  base:
    - /srv/salt-repo/salt/dev

fileserver_backend:
  - roots

pillar_roots:
  base:
    - /srv/salt-repo/pillar/dev

peer_run:
  .*:
      - vault.generate_token
include:
  - /config/secrets.conf

Steps to Reproduce the behavior

  1. Create a minion with some grains
  2. Run cache.grains function targeting the new minion to cache grains
  3. Verify data is cached by checking cachedir on localfs (in my case it is /var/cache/salt)
  4. Run salt-run cache.fetch command to fetch cached grains (in my case salt-run cache.fetch minions/temp-0572/data grains temp-0572 being the minion id)
  5. Make a HTTP request for the same function:
curl -XPOST http://localhost:8000 -d '{"client": "runner", "fun": "cache.fetch", "bank": "minions/temp-0572/data", "key": "grains", "cachedir": "/var/cache/salt"}'

Then see the request failed with following error:

Passed invalid arguments: fetch() missing 1 required positional argument: 'key'

Usage:

    Fetch data from a salt.cache bank.

    CLI Example:

    .. code-block:: bash

        salt-run cache.fetch
cloud/active/ec2/myec2 myminion cachedir=/var/cache/salt/

Expected behavior
Have the same successful result as step 4 on step 5.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
          Salt: 3003

Dependency Versions:
          cffi: 1.14.4
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10.1
       libgit2: 1.1.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.17
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: 1.6.1
        Python: 3.7.12 (default, Sep  8 2021, 02:04:31)
  python-gnupg: 0.4.4
        PyYAML: 5.3.1
         PyZMQ: 18.0.1
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.1

System Versions:
          dist: alpine 3.13.6
        locale: UTF-8
       machine: x86_64
       release: 5.4.129-63.229.amzn2.x86_64
        system: Linux
       version: Alpine Linux 3.13.6

Additional context
I can use HTTP API without any problem except for this case. I tried different payloads with the request, it didn't matter I always got the same response:

'{"client": "runner", "fun": "cache.fetch", "bank": "minions/temp-0572/data", "key": "grains", "cachedir": "/var/cache/salt"}'
'{"client": "runner", "fun": "cache.fetch", "bank": "minions/temp-0572/data", "key": "grains"}'
'{"client": "runner", "fun": "cache.fetch", "key": "grains", "bank": "minions/temp-0572/data"}'
[...]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the rest_cherrypy request path for runner calls and the cache.fetch entry point, comparing the HTTP argument handling with the working salt-run cache.fetch invocation. Reproduce the request using the payloads in the issue and trace why key is not received. Done means the HTTP request returns the same cached grains as the CLI command.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.