E3SM-Project / E3SM-Project/zstash
[Feature]: Improve zstash testing
- Dominant language
- Python
- Stars
- 11
- Forks
- 10
- Avg merge
- 14h 18m
- Merged PRs (30d)
- 1
Description
### How will this affect the next version number?
New feature (increment MINOR version)
### Is your feature request related to a problem?
Current testing has a number of problems:
- So-called "unit" testing is really more integration testing because it involves I/O rather than pure function testing. This leads to two points:
- It may perhaps be better to convert the "unit" tests to bash scripts that check output. This would allow more transparency in what workflows are being tested. For example, one implementation of this is `tests/scripts/globus_auth.bash`.
- Actual unit tests should test functions used by `zstash` rather than the actual `zstash` commands called from the command line.
- The current test framework uses `unittest` rather than `pytest`. I recall trying to modernize that in the past, but running into problems.
- The current test framework always uses the GitHubActions or `conda/dev.yml` Python version. We should simulate running `zstash create` with an older version of Python and/or `zstash` and then running `zstash extract` on a later version. For example, see the thread at https://github.com/E3SM-Project/zstash/pull/384#discussion_r2389348376. It's a common use case that someone would be extracting data that was archived a while ago.
### Describe the solution you'd like
Described above.
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the current tests and the example workflow in tests/scripts/globus_auth.bash, then inspect the unittest setup and the GitHub Actions or conda/dev.yml Python versions. Read the discussion on pull request 384 for the compatibility case involving zstash create and extract. Done means the testing approach, framework, and older-to-newer version coverage described in the issue are implemented and passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, github-actions, python
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100