E3SM-Project / E3SM-Project/zstash

[Feature]: Improve zstash testing

Open
#385 4 comments 0 reactions 0 assignees View on GitHub
Testing
Dominant language
Python
Stars
11
Forks
10
Avg merge
14h 18m
Merged PRs (30d)
1

Description

### How will this affect the next version number?

New feature (increment MINOR version)

### Is your feature request related to a problem?

Current testing has a number of problems:
- So-called "unit" testing is really more integration testing because it involves I/O rather than pure function testing. This leads to two points:
- It may perhaps be better to convert the "unit" tests to bash scripts that check output. This would allow more transparency in what workflows are being tested. For example, one implementation of this is `tests/scripts/globus_auth.bash`.
- Actual unit tests should test functions used by `zstash` rather than the actual `zstash` commands called from the command line.
- The current test framework uses `unittest` rather than `pytest`. I recall trying to modernize that in the past, but running into problems.
- The current test framework always uses the GitHubActions or `conda/dev.yml` Python version. We should simulate running `zstash create` with an older version of Python and/or `zstash` and then running `zstash extract` on a later version. For example, see the thread at https://github.com/E3SM-Project/zstash/pull/384#discussion_r2389348376. It's a common use case that someone would be extracting data that was archived a while ago.

### Describe the solution you'd like

Described above.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the current tests and the example workflow in tests/scripts/globus_auth.bash, then inspect the unittest setup and the GitHub Actions or conda/dev.yml Python versions. Read the discussion on pull request 384 for the compatibility case involving zstash create and extract. Done means the testing approach, framework, and older-to-newer version coverage described in the issue are implemented and passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
bash, github-actions, python
Domain
ci-cd, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.