Remove test suite outcome dependence on run directory
Nessuno ha ancora preso questa issue.
- Lingua principale
- Python
- Stelle
- 150
- Fork
- 23
- Merge medio
- 1g 11h
- PR unite (30g)
- 2
Descrizione
At the moment the pass/fail outcome of the test suite depends on the directory which the suite script run_test.py is run from. That is, certain test modules & hence the full suite seem to have a dependence on the initial run directory.
As a demonstration, if the full test suite passes when run from the cf-python/cf/test repo where the tests live, it will fail when run from some outside directory, but not in a consistent way given the specific outside directory. For example, from the root directory repo I observe that the unit test loading errors:
$ pwd
/home/sadie/cf-python
$ python cf/test/run_tests.py
--------------------
CF-PYTHON TEST SUITE
--------------------
Run date: 2020-07-17 01:55:20.056541
Platform: Linux-4.15.0-54-generic-x86_64-with-debian-buster-sid
HDF5 library: 1.10.4
netcdf library: 4.6.3
udunits2 library: /home/sadie/anaconda3/lib/libudunits2.so.0
python: 3.7.6 /home/sadie/anaconda3/bin/python
netCDF4: 1.5.3 /home/sadie/anaconda3/lib/python3.7/site-packages/netCDF4/__init__.py
cftime: 1.1.3 /home/sadie/anaconda3/lib/python3.7/site-packages/cftime/__init__.py
numpy: 1.18.1 /home/sadie/anaconda3/lib/python3.7/site-packages/numpy/__init__.py
psutil: 5.7.0 /home/sadie/anaconda3/lib/python3.7/site-packages/psutil/__init__.py
scipy: 1.4.1 /home/sadie/anaconda3/lib/python3.7/site-packages/scipy/__init__.py
matplotlib: 3.1.3 /home/sadie/anaconda3/lib/python3.7/site-packages/matplotlib/__init__.py
ESMF: 8.0.1 /home/sadie/anaconda3/lib/python3.7/site-packages/ESMF/__init__.py
cfdm: 1.8.6 /home/sadie/cfdm/cfdm/__init__.py
cfunits: 3.2.9 /home/sadie/cfunits/cfunits/__init__.py
cfplot: 3.0.6 /home/sadie/anaconda3/lib/python3.7/site-packages/cfplot/__init__.py
cf: 3.6.0 /home/sadie/cf-python/cf/__init__.py
Running tests from /home/sadie/cf-python
cf.abstract (unittest.loader._FailedTest) ... ERROR
======================================================================
ERROR: cf.abstract (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: cf.abstract
Traceback (most recent call last):
File "/home/sadie/anaconda3/lib/python3.7/unittest/loader.py", line 470, in _find_test_path
package = self._get_module_from_name(name)
File "/home/sadie/anaconda3/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "/home/sadie/cf-python/cf/abstract/__init__.py", line 1, in <module>
from .coordinate import Coordinate
ImportError: cannot import name 'Coordinate' from 'cf.abstract.coordinate' (/home/sadie/cf-python/cf/abstract/coordinate.py)
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (errors=1)
cf.abstract (unittest.loader._FailedTest) ... ERROR
test_create_field (cf.test.setup_create_field.create_fieldTest) ... ok
======================================================================
ERROR: cf.abstract (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: cf.abstract
Traceback (most recent call last):
File "/home/sadie/anaconda3/lib/python3.7/unittest/loader.py", line 470, in _find_test_path
package = self._get_module_from_name(name)
File "/home/sadie/anaconda3/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "/home/sadie/cf-python/cf/abstract/__init__.py", line 1, in <module>
from .coordinate import Coordinate
ImportError: cannot import name 'Coordinate' from 'cf.abstract.coordinate' (/home/sadie/cf-python/cf/abstract/coordinate.py)
----------------------------------------------------------------------
Ran 2 tests in 0.456s
FAILED (errors=1)
test_CellMeasure_identity (cf.test.test_CellMeasure.CellMeasureTest) ... ok
...
...
...
test_manage_log_level_via_verbose_attr (cf.test.test_decorators.DecoratorsTest) ... ok
======================================================================
ERROR: cf.abstract (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: cf.abstract
Traceback (most recent call last):
File "/home/sadie/anaconda3/lib/python3.7/unittest/loader.py", line 470, in _find_test_path
package = self._get_module_from_name(name)
File "/home/sadie/anaconda3/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "/home/sadie/cf-python/cf/abstract/__init__.py", line 1, in <module>
from .coordinate import Coordinate
ImportError: cannot import name 'Coordinate' from 'cf.abstract.coordinate' (/home/sadie/cf-python/cf/abstract/coordinate.py)
----------------------------------------------------------------------
Ran 275 tests in 651.759s
FAILED (errors=1)
& outside of the repo directory completely the run_tests.py script picks up on & runs zero tests, notably implying a pass when this may not be the case. For instance, from my $HOME:
$ pwd
/home/sadie
$ ls ~/cf-python/cf/test/run_tests.py
/home/sadie/cf-python/cf/test/run_tests.py
$ python cf-python/cf/test/run_tests.py
--------------------
CF-PYTHON TEST SUITE
--------------------
Run date: 2020-07-17 01:26:43.800432
Platform: Linux-4.15.0-54-generic-x86_64-with-debian-buster-sid
HDF5 library: 1.10.4
netcdf library: 4.6.3
udunits2 library: /home/sadie/anaconda3/lib/libudunits2.so.0
python: 3.7.6 /home/sadie/anaconda3/bin/python
netCDF4: 1.5.3 /home/sadie/anaconda3/lib/python3.7/site-packages/netCDF4/__init__.py
cftime: 1.1.3 /home/sadie/anaconda3/lib/python3.7/site-packages/cftime/__init__.py
numpy: 1.18.1 /home/sadie/anaconda3/lib/python3.7/site-packages/numpy/__init__.py
psutil: 5.7.0 /home/sadie/anaconda3/lib/python3.7/site-packages/psutil/__init__.py
scipy: 1.4.1 /home/sadie/anaconda3/lib/python3.7/site-packages/scipy/__init__.py
matplotlib: 3.1.3 /home/sadie/anaconda3/lib/python3.7/site-packages/matplotlib/__init__.py
ESMF: 8.0.1 /home/sadie/anaconda3/lib/python3.7/site-packages/ESMF/__init__.py
cfdm: 1.8.6 /home/sadie/cfdm/cfdm/__init__.py
cfunits: 3.2.9 /home/sadie/cfunits/cfunits/__init__.py
cfplot: 3.0.6 /home/sadie/anaconda3/lib/python3.7/site-packages/cfplot/__init__.py
cf: 3.6.0 /home/sadie/cf-python/cf/__init__.py
Running tests from /home/sadie
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
We should remove all coupling between run directory and test module outcome.
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Direzione di ricerca
Inizia riproducendo i risultati differenti dal repository, dalla sua root e da $HOME usando cf/test/run_tests.py. Esamina la discovery dei test e la gestione dei percorsi in quello script, quindi verifica che venga selezionata la stessa suite indipendentemente dalla directory di lavoro. Il lavoro è completato quando i test non falliscono a causa della directory di avvio e una discovery vuota non può segnalare un successo fuorviante.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python
- Ambito
- testing-qa
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100