envoyproxy / envoyproxy/nighthawk

asserts.py module does not appear to get packaged in the benchmark container

Open
#538 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
414
Forks
95
Avg merge
2d 9h
Merged PRs (30d)
11

Description

If I execute the fully dockerized benchmark and use `test_discovery.py` as a discovered test, it fails due to the inability to import the asserts module

Repro steps:

Using the flow outlined in the [fully dockerized flow](https://github.com/envoyproxy/nighthawk/blob/master/benchmarks/README.md#example-fully-dockerized-flow)
- I set `${TEST_DIR}` to `${HOME}/nighthawk/benchmarks/test`

Running the docker container produces the following error:

```
______________________________________________________________________________________________________________ ERROR collecting external_tests/test_discovery.py _______________________________________________________________________________________________________________
ImportError while importing test module '/usr/local/bin/benchmarks/benchmarks.runfiles/nighthawk/benchmarks/external_tests/test_discovery.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
benchmarks.runfiles/python_pip_deps_pypi__pytest_5_4_2/_pytest/python.py:511: in _importtestmodule
mod = self.fspath.pyimport(ensuresyspath=importmode)
benchmarks.runfiles/python_pip_deps_pypi__py_1_8_1/py/_path/local.py:701: in pyimport
__import__(modname)
benchmarks.runfiles/python_pip_deps_pypi__pytest_5_4_2/_pytest/assertion/rewrite.py:152: in exec_module
exec(co, module.__dict__)
benchmarks.runfiles/nighthawk/benchmarks/external_tests/test_discovery.py:14: in
from test.integration import asserts
E ImportError: cannot import name 'asserts' from 'test.integration' (/usr/local/bin/benchmarks/benchmarks.runfiles/nighthawk/test/integration/__init__.py)
```

If I modify `test_discovery.py` and comment out all uses of "asserts", ultimately I see a successful run:

```
======================= test session starts =======================
platform linux -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python3
rootdir: /usr/local/bin/benchmarks/benchmarks.runfiles/nighthawk/benchmarks
plugins: dependency-0.5.1, xdist-1.34.0
collected 5 items

benchmarks.runfiles/nighthawk/benchmarks/external_tests/test_discovery.py::test_http_h1_small_request_small_reply_via[IpVersion.IPV4-nighthawk/test/integration/configurations/nighthawk_http_origin.yaml-nighthawk/benchmarks/configurations/envoy_proxy.yaml]

...
=================== 5 passed in 185.60s (0:03:05) ====================
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.