Deltares / Deltares/imod-python

Speed up unittests

Open
#1,467 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ci
Dominant language
Python
Stars
41
Forks
12
Avg merge
21h 8m
Merged PRs (30d)
1

Description

I'm a bit annoyed by the long time of our unittests, especially on TeamCity.
The from_imod5_data tests we have seem to be very slow. Probably worth some investigation.
Running our unittests locally with --duration=50 prints:

75.13s call     imod/tests/test_mf6/test_mf6_wel.py::test_import_and_convert_to_mf6[Well]
65.38s call     imod/tests/test_mf6/test_mf6_hfb.py::test_hfb_from_imod5
59.48s call     imod/tests/test_mf6/test_mf6_drn.py::test_from_imod5[default]
58.78s call     imod/tests/test_mf6/test_mf6_regrid_model.py::test_model_regridding_can_skip_validation
58.78s call     imod/tests/test_mf6/test_mf6_riv.py::test_import_river_from_imod5
58.02s call     imod/tests/test_mf6/test_mf6_wel.py::test_import_and_convert_to_mf6[LayeredWell]
49.52s call     imod/tests/test_mf6/test_mf6_drn.py::test_from_imod5[custom]
48.75s call     imod/tests/test_mf6/test_mf6_riv.py::test_import_river_from_imod5__negative_layer
48.09s call     imod/tests/test_mf6/test_mf6_regrid_model.py::test_model_regridding_can_validate
47.94s call     imod/tests/test_mf6/test_mf6_chd.py::test_from_imod5
47.69s call     imod/tests/test_mf6/test_mf6_chd.py::test_from_imod5_shd
42.67s setup    imod/tests/test_mf6/test_mf6_rch.py::test_planar_rch_from_imod5_constant
39.42s call     imod/tests/test_mf6/test_mf6_drn.py::test_from_imod5__negative_layer
38.98s call     imod/tests/test_mf6/test_mf6_riv.py::test_import_river_from_imod5__infiltration_factors
38.24s call     imod/tests/test_mf6/test_mf6_riv.py::test_import_river_from_imod5__period_data
37.24s call     imod/tests/test_mf6/test_mf6_generalheadboundary.py::test_from_imod5_planar
37.21s call     imod/tests/test_mf6/test_mf6_generalheadboundary.py::test_from_imod5_non_planar
35.37s setup    imod/tests/test_mf6/test_mf6_wel.py::test_import_and_convert_to_mf6[Well]
31.77s call     imod/tests/test_mf6/test_mf6_rch.py::test_planar_rch_from_imod5_transient
31.03s call     imod/tests/test_mf6/test_mf6_rch.py::test_planar_rch_from_imod5_constant
30.82s call     imod/tests/test_mf6/test_mf6_rch.py::test_non_planar_rch_from_imod5_constant
29.59s setup    imod/tests/test_mf6/test_mf6_chd.py::test_from_imod5
28.04s call     imod/tests/test_mf6/test_mf6_dis.py::test_from_imod5_data__validation_error
24.86s call     imod/tests/test_mf6/test_mf6_wel.py::test_import_and_cleanup[Well]
23.19s call     imod/tests/test_mf6/test_mf6_rch.py::test_non_planar_rch_from_imod5_transient
23.03s call     imod/tests/test_mf6/test_mf6_dis.py::test_from_imod5_data__write
22.72s call     imod/tests/test_mf6/test_mf6_dis.py::test_from_imod5_data__idomain_values
22.11s call     imod/tests/test_mf6/test_mf6_dis.py::test_from_imod5_data__grid_extent
22.06s call     imod/tests/test_mf6/test_mf6_rch.py::test_from_imod5_cap_data
14.98s setup    imod/tests/test_mf6/test_mf6_sto.py::test_from_imod5
11.26s call     imod/tests/test_mf6/test_multimodel/test_mf6_partitioning_unstructured.py::test_partition_transport_multispecies[three_parts]
10.46s call     imod/tests/test_mf6/test_mf6_mask_simulation.py::test_mask_simulation
10.35s setup    imod/tests/test_mf6/test_mf6_drn.py::test_from_imod5[default]
10.00s setup    imod/tests/test_mf6/test_mf6_hfb.py::test_hfb_from_imod5
9.40s call     imod/tests/test_mf6/test_mf6_regrid_transport.py::test_regrid_transport[col_row_dimension2]
9.19s setup    imod/tests/test_mf6/test_mf6_riv.py::test_import_river_from_imod5
8.97s call     imod/tests/test_mf6/test_multimodel/test_mf6_partitioning_unstructured.py::test_partition_transport_multispecies[two_parts]
8.76s call     imod/tests/test_mf6/test_multimodel/test_mf6_partitioning_unstructured.py::test_partition_transport[three_parts]
8.33s call     imod/tests/test_mf6/test_multimodel/test_mf6_modelsplitter_transport.py::test_split_flow_and_transport_model_evaluate_output_with_species
7.86s call     imod/tests/test_mf6/test_multimodel/test_mf6_modelsplitter_transport.py::test_split_flow_and_transport_model_evaluate_output
7.31s call     imod/tests/test_mf6/test_multimodel/test_mf6_partitioning_unstructured.py::test_partition_transport_multispecies[concentric]
7.22s call     imod/tests/test_mf6/test_mf6_regrid_transport.py::test_regrid_transport[col_row_dimension0]
6.41s setup    imod/tests/test_mf6/test_mf6_ic.py::test_from_imod5
6.35s call     imod/tests/test_mf6/test_multimodel/test_mf6_modelsplitter_transport.py::test_split_dump
6.07s call     imod/tests/test_mf6/test_multimodel/test_mf6_modelsplitter_transport.py::test_split_flow_and_transport_model
6.03s setup    imod/tests/test_mf6/test_mf6_generalheadboundary.py::test_from_imod5_non_planar
5.99s call     imod/tests/test_mf6/test_mf6_regrid_transport.py::test_regrid_transport[col_row_dimension1]
5.88s setup    imod/tests/test_mf6/test_mf6_riv.py::test_import_river_from_imod5__period_data
5.50s setup    imod/tests/test_mf6/test_mf6_dis.py::test_from_imod5_data__idomain_values
5.46s call     imod/tests/test_mf6/test_multimodel/test_mf6_partitioning_unstructured.py::test_partition_transport[concentric]

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the unittest suite with --duration=50 and inspect the listed slow tests under imod/tests/test_mf6/, especially the from_imod5 tests and regridding tests. Compare setup and call times to identify where investigation should focus; done means the relevant unittest runtime is reduced and the affected tests still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, testing-qa
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.