AcademySoftwareFoundation / AcademySoftwareFoundation/rez
rez-test --interactive flag
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 374
- Avg merge
- 9d 12h
- Merged PRs (30d)
- 5
Description
I've been using [a custom script](https://github.com/ColinKennedy/rez_developer_packages/tree/master/rez_test_env) to get "a Rez environment of a rez-test" for a while but it has flaws. Looking through Rez's list of issues and PRs, it looks like this has been on the menu for a while (https://github.com/nerdvegas/rez/issues/665).
I'd like to implement this work. Full disclosure, I've already written a mostly-working version of this, including unittests ([in this forked branch](https://github.com/nerdvegas/rez/compare/master...ColinKennedy:issues/665-add_rez_test_interactive_option?expand=1)). But before putting in a PR, I wanted to discuss it a bit more. Reading Fede's PR on the topic (https://github.com/nerdvegas/rez/pull/759) was very insightful. However I discovered a some sharp edges related to tests "requires", "on_variants" and the UX of providing an explicit `--variant` (or not). I think I got a list of expected behavior that is consistent and makes sense though. I'll outline it below.
Given `some_package`:
```python
name = "some_package"
# ... more package definition here ...
variants = [["python-2", "backports.functools-lru-cache-1"], ["python-3"]]
tests = {
"foo": {
"command": "echo foo",
"requires": ["python-2", "six"],
},
"fizz": {
"command": "echo another",
"requires": ["python-2", "extra_package-2"],
},
"bar": {
"command": "echo foo",
"requires": ["python-3"],
},
"buzz": {
"command": "echo buzz",
"requires": ["python-3", "thing-1"],
},
"lastly": {
"command": "echo lastly",
"requires": ["python-3"],
"run_on": "explicit",
}
}
```
Here's how I'm thinking that the CLI would behave:
```
# Gets the combination of "foo & fizz"
rez-test some_package foo fizz --interactive
# Resolves the "bar & buzz" package. "lastly" is ignored because it is "explicit"
rez-test some_package --variant 1 --interactive
# Resolves with "lastly", explicitly
rez-test some_package lastly --interactive
# Fails to resolve because --variant 0 does not match the requires of "lastly"
rez-test some_package lastly --variant 0 --interactive
# Fails to resolve because "foo / fizz" conflict with "bar", which are all default tests
rez-test some_package --interactive
# Fail to resolve because python versions conflict
rez-test some_package foo bar --interactive
# Resolves if all default tests are compatible
rez-test another_package --interactive
```
To summarize the behavior, I tried to make `--interactive` work like the actual `rez-env` behavior.
- If `--variant` is included, the resolve must use that variant or fail
- If a test is included explicitly, its requirements / content must be included in the resolve or fail
And the flipside also being true
- If no `--variant` is given (and the package has a variant), a suitable resolve using any variant is returned
I'm considering also adding behavior such as "if no test names are given, not all of the default tests are guaranteed to be part of the resolve". Because it could be that not all tests are resolvable and maybe variant "0" resolves 2/4 tests but variant 1 resolves 3/4 tests. In either case, you're missing at least one test but at least you get a resolve back. The logic does not do this currently, but that may be a consideration. Maybe people will think that's a good idea.
Special consideration is given for "on_variants". For example if 2+ tests have
a common variant that resolves and the user doesn't specify an explicit
`--variant`, a matching resolve is returned. However if they *do* specify
`--variant` and it is invalid, the resolve conflicts and fails.
What do you think of this behavior? I can make a PR with the changes as-is but I figured it'd be better to discuss the exact behavior in a discussion thread prior to opening a PR, in case others have suggestions.
Contributor guide
Research direction
Start by tracing the rez-test CLI behavior for --interactive, --variant, test requires, run_on, and on_variants, then compare it with rez-env behavior and the linked fork's implementation and unittests. Done means the documented combinations resolve or fail consistently, including explicit tests and variants, with the relevant tests passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100