OpenMDAO / OpenMDAO/Aviary

Investigate slow tests and benchmarks.

Open
#1,105 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

performance
Dominant language
Python
Stars
287
Forks
140
Avg merge
6d 14h
Merged PRs (30d)
18

Description

Some tests are taking longer than reasonable. Longer tests should probably be in the benchmark set instead. Should look into why some of these are optimizing so poorly.

testflo supports timing with testflo --duration 25 to report the 25 tests with the longest execution time.

137.882 sec - /validation_cases/benchmark_tests/test_problem_types_GwGm.py:TestOffDesign.test_off_design_SNOPT
  63.905 sec - /validation_cases/benchmark_tests/test_problem_types_GwGm.py:TestOffDesign.test_off_design_IPOPT
  55.545 sec - /validation_cases/benchmark_tests/test_bwb_GwGm.py:ProblemPhaseTestCase.test_bench_bwb_GwGm_SNOPT
  52.506 sec - /interface/test/test_height_energy_mission.py:AircraftMissionTestSuite.test_mission_distance_solve_segments_IPOPT
  40.157 sec - /validation_cases/benchmark_tests/test_bench_off_design.py:Test2DOFOffDesign.test_alternate_mission_match
  36.041 sec - /validation_cases/benchmark_tests/test_bench_off_design.py:Test2DOFOffDesign.test_fallout_mission_changed
  34.981 sec - /validation_cases/benchmark_tests/test_bench_off_design.py:Test2DOFOffDesign.test_fallout_mission_match
  34.799 sec - /validation_cases/benchmark_tests/test_bench_off_design.py:Test2DOFOffDesign.test_alternate_mission_changed
  32.770 sec - /subsystems/aerodynamics/gasp_based/test/test_gaspaero.py:GASPAeroTest.test_ground
  32.315 sec - /validation_cases/benchmark_tests/test_bench_multimission.py:MultiMissionTestcase.test_multimission
  26.396 sec - /validation_cases/benchmark_tests/test_battery_in_a_mission.py:TestBatteryMission.test_subsystems_in_a_mission_2dof
  25.029 sec - /validation_cases/benchmark_tests/test_bench_GwGm.py:ProblemPhaseTestCase.test_bench_GwGm_IPOPT_Breguet_Cruise
  24.926 sec - /validation_cases/benchmark_tests/test_bench_GwGm.py:ProblemPhaseTestCase.test_bench_GwGm_IPOPT
  21.025 sec - /interface/test/test_height_energy_mission.py:AircraftMissionTestSuite.test_mission_distance_solve_segments_SLSQP
  20.829 sec - /validation_cases/benchmark_tests/test_bench_off_design.py:PayloadRangeTest.test_payload_range
  20.204 sec - /interface/test/test_reports.py:TestReports.test_multiple_off_design_report_directories
  19.358 sec - /validation_cases/benchmark_tests/test_bench_FwFm.py:TestBenchFwFmSerial.test_bench_FwFm_SNOPT
  19.177 sec - /validation_cases/benchmark_tests/test_bench_GwGm.py:ProblemPhaseTestCase.test_bench_GwGm_SNOPT
  17.323 sec - /validation_cases/benchmark_tests/test_bench_FwFm.py:TestBenchFwFmSerial.test_bench_FwFm_IPOPT
  16.872 sec - /validation_cases/benchmark_tests/test_bench_large_turboprop_freighter.py:LargeTurbopropFreighterBenchmark.test_bench_2DOF
  16.810 sec - /subsystems/mass/gasp_based/test/test_fixed.py:FixedMassGroupTestCase3.test_case1
  14.304 sec - /interface/test/test_save_results.py:TestSizingResults.test_alternate
  13.162 sec - /validation_cases/benchmark_tests/test_bench_off_design.py:TestEnergyStateOffDesign.test_fallout_mission_changed
  13.067 sec - /interface/test/test_height_energy_mission.py:AircraftMissionTestSuite.test_mission_optimize_altitude_and_mach_shooting
  13.056 sec - /interface/test/test_height_energy_mission.py:AircraftMissionTestSuite.test_mission_basic_shooting_pyopt

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with testflo --duration 25 and the slow tests listed in the issue, especially the cases under validation_cases/benchmark_tests/ and the slower tests under interface/test and subsystems. Inspect why these tests take so long and whether they belong in the benchmark set. Done means the slow-test causes and any appropriate test or benchmark reorganization are identified and addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, testing-qa
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.