indygreg / indygreg/PyOxidizer

test subcommand

Open
#141 9 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
6.2k
Forks
256
PR merge metrics
No merged PRs in 30d

Description

It would be nice if pyozidizer had a test command that was the equivalent of `setup.py test` (I know setuptools is trying to kill it, and that it will only work for a subset of packages, but ~50% of the core packages provided by a Unix distro will support it. And adding an equivalent of `setup.py pytest` would provide much higher compatibility)

A very useful part of Nuitka is that it can build Python packages as loadable shared libraries. This means that each dependent package can then be tested using the normal test suite of the package, using whatever test runner is needed, such as nose or pytest, and the test runner doesnt need to have been compiled with Nuitka. This doesnt guarantee that a standalone built with Nuitka would also pass the test suite of those packages, but it is a helpful smoke test to catch dependencies doing something strange in a new release which would break the standalone in a hard to QA / detect way.

I can achieve an even better result with PyOxidizer with some fiddling. e.g. the following relevant part of `pyoxidizer.toml`

```toml
[[packaging_rule]]
type = "pip-install-simple"
package = "pytz"

# Add tests which are not installed
[[packaging_rule]]
type = "package-root"
path = "/path/to/sourcetree/pytz-2019.3"
packages = ["pytz.tests"]
```

```py
$ /path/to/pytz
>>> import unittest
>>> unittest.main('pytz.tests.test_lazy')
.......................................
----------------------------------------------------------------------
Ran 39 tests in 0.010s

OK

$ /path/to/pytz
>>> import unittest
>>> unittest.main('pytz.tests.test_tzinfo')
Traceback (most recent call last):
File "", line 1, in
File "unittest.main", line 70, in __init__
File "pytz.tests.test_tzinfo", line 40, in
File "pytz", line 172, in timezone
File "pytz", line 198, in _case_insensitive_zone_lookup
File "pytz.lazy", line 101, in _lazy
File "pytz", line 1104, in
File "pytz", line 114, in resource_exists
File "pytz", line 95, in open_resource
NameError: name '__file__' is not defined
```

boom. There is the failure in https://github.com/indygreg/PyOxidizer/issues/91 .

Roughly the same toml is needed for `appdirs`, excepting its tests are in a `test` package which isnt installed.

If that could be wrapped up into a test command, it would be easier to determine which packages are likely broken when built with PyOxidizer, and also provide a way to quickly verify that those bugs have been fixed.

https://github.com/indygreg/PyOxidizer/issues/122 gave me another idea -- it would be useful to be able to build an exe which has an entry point that is for the app CLI, but then run something like `USE_PYTHON_CLI=1 my_new_exe -m unittest` or similar to invoke a hidden CLI which is a close replica of the CPython cli, thereby getting better access into the internals of the built exe.

Attempting with nose failed due to https://github.com/indygreg/PyOxidizer/issues/139

Attempting with pytest failed due to https://github.com/indygreg/PyOxidizer/issues/140

Contributor guide

No contributing guide indexed for this repository

Research direction

No source files or tests are named. Start by reviewing the proposed pyoxidizer.toml packaging rules and the desired setup.py test/pytest and hidden Python CLI behavior, then read the referenced issues #91, #122, #139, and #140. Done means the requested test workflow and CLI access are defined and verified against dependency test suites.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
build-system, cli, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.