splunk / splunk/splunk-platform-automator

Harden spa CLI argv tests (flag matrix + isolated env)

Open
#89 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
137
Forks
51
Avg merge
7h 44m
Merged PRs (30d)
14

Description

Summary

spa argument parsing is split across a few helpers (_split_passthrough, _split_native, _run_help_target, native-flag commands, aliases, -- extra Ansible args, --agent / --json / --no-agent). Existing tests cover a handful of cases (tests/test_spa_cli.py, tests/test_spa_cli_contract.py) but they are not a contract for “this argv always means this command + these forwarded flags.”

Two failure modes showed up while iterating on validate/playbooks:

  1. Real parser gaps (flags after the subcommand, aliases vs canonical help, -- passthrough, native tools like spa shell / spa aws --json) only fail if someone wrote a test for that exact shape.
  2. False failures when the operator’s shell has SPA_* / ANSIBLE_* / SPLUNK_CONFIG_FILE exported (eval "$(spa env --export)"). spa_env() only strips agent-detect vars, so ./tests/run_local_tests.sh inherits those and help/CLI tests can fail even though the parser is fine.

Ship

  1. Isolate subprocess env in tests/spa_testutil.py spa_env() (and run_spa / pytest autouse if needed): drop inherited SPA_HOME, SPA_ENV_DIR, SPA_VENV_DIR, SPA_SOFTWARE_DIR, SPA_BASECONFIG_DIR, SPA_APPS_DIR, SPLUNK_CONFIG_FILE, and Ansible plugin/config/inventory vars unless a test sets them. ./tests/run_local_tests.sh must be hermetic from a dirty operator shell.
  2. Table-driven argv contract (pytest, @pytest.mark.local):
    • Every top-level command and alias from spa agent schema (or the argparse canonical map): spa ALIAS --help prints usage: spa CANONICAL.
    • Globals before and after the subcommand: --json, --agent, --no-agent, --yes, --env, --start-dir, -v where accepted.
    • Native-flag commands (shell/sh, aws, licenses/lic): spa does not eat tool flags; --json on aws/licenses stays native unless --agent.
    • spa run NAME --yes -- --help (and other -- extras) must reach Ansible extras, not spa help.
    • --hosts / --apps-playbook value flags are not mistaken for playbook names.
  3. Prefer unit tests of the splitters (in-process, no bin/spa) plus a smaller bin/spa smoke matrix. Assert exact forwarded argv, not only “no unrecognized arguments.”
  4. Generate alias/command rows from the schema so the table cannot drift when a command is added.

Out of scope

Rewriting argparse. This is tests + env isolation; parser changes only if a row proves a real bug.

Related

  • Parser: lib/spa/cli.py (_split_passthrough, _split_native, _run_help_target, NATIVE_FLAG_COMMANDS)
  • Tests: tests/spa_testutil.py, tests/test_spa_cli.py, tests/test_spa_cli_contract.py, tests/conftest.py

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/spa_testutil.py and the existing cases in tests/test_spa_cli.py and tests/test_spa_cli_contract.py, then read the splitter entry points in lib/spa/cli.py. Run ./tests/run_local_tests.sh and inspect tests/conftest.py for environment setup. Done means isolated subprocess environments and table-driven assertions cover the listed command, alias, global-flag, native-flag, passthrough, and value-flag cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
ansible, python
Domain
cli, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.