splunk / splunk/splunk-platform-automator
Harden spa CLI argv tests (flag matrix + isolated env)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 137
- Forks
- 51
- Avg merge
- 7h 44m
- Merged PRs (30d)
- 14
Description
Summary
spa argument parsing is split across a few helpers (_split_passthrough, _split_native, _run_help_target, native-flag commands, aliases, -- extra Ansible args, --agent / --json / --no-agent). Existing tests cover a handful of cases (tests/test_spa_cli.py, tests/test_spa_cli_contract.py) but they are not a contract for “this argv always means this command + these forwarded flags.”
Two failure modes showed up while iterating on validate/playbooks:
- Real parser gaps (flags after the subcommand, aliases vs canonical help,
--passthrough, native tools likespa shell/spa aws --json) only fail if someone wrote a test for that exact shape. - False failures when the operator’s shell has
SPA_*/ANSIBLE_*/SPLUNK_CONFIG_FILEexported (eval "$(spa env --export)").spa_env()only strips agent-detect vars, so./tests/run_local_tests.shinherits those and help/CLI tests can fail even though the parser is fine.
Ship
- Isolate subprocess env in
tests/spa_testutil.pyspa_env()(andrun_spa/ pytest autouse if needed): drop inheritedSPA_HOME,SPA_ENV_DIR,SPA_VENV_DIR,SPA_SOFTWARE_DIR,SPA_BASECONFIG_DIR,SPA_APPS_DIR,SPLUNK_CONFIG_FILE, and Ansible plugin/config/inventory vars unless a test sets them../tests/run_local_tests.shmust be hermetic from a dirty operator shell. - Table-driven argv contract (pytest,
@pytest.mark.local):- Every top-level command and alias from
spa agent schema(or the argparsecanonicalmap):spa ALIAS --helpprintsusage: spa CANONICAL. - Globals before and after the subcommand:
--json,--agent,--no-agent,--yes,--env,--start-dir,-vwhere accepted. - Native-flag commands (
shell/sh,aws,licenses/lic): spa does not eat tool flags;--jsonon aws/licenses stays native unless--agent. spa run NAME --yes -- --help(and other--extras) must reach Ansible extras, not spa help.--hosts/--apps-playbookvalue flags are not mistaken for playbook names.
- Every top-level command and alias from
- Prefer unit tests of the splitters (in-process, no
bin/spa) plus a smallerbin/spasmoke matrix. Assert exact forwarded argv, not only “nounrecognized arguments.” - Generate alias/command rows from the schema so the table cannot drift when a command is added.
Out of scope
Rewriting argparse. This is tests + env isolation; parser changes only if a row proves a real bug.
Related
- Parser:
lib/spa/cli.py(_split_passthrough,_split_native,_run_help_target,NATIVE_FLAG_COMMANDS) - Tests:
tests/spa_testutil.py,tests/test_spa_cli.py,tests/test_spa_cli_contract.py,tests/conftest.py
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/spa_testutil.py and the existing cases in tests/test_spa_cli.py and tests/test_spa_cli_contract.py, then read the splitter entry points in lib/spa/cli.py. Run ./tests/run_local_tests.sh and inspect tests/conftest.py for environment setup. Done means isolated subprocess environments and table-driven assertions cover the listed command, alias, global-flag, native-flag, passthrough, and value-flag cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ansible, python
- Domain
- cli, testing
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100