OpenMDAO / OpenMDAO/Aviary

Fix tests that do not converge

Open
#881 2 comments 0 reactions 5 assignees View on GitHub

@xjjiang is already working on this.

Since Jul 13, 2026.

validation
Dominant language
Python
Stars
287
Forks
140
Avg merge
6d 14h
Merged PRs (30d)
18

Description

Description

We need to add in checks to all the benchmark tests to ensure that the problem converges successfully before comparing expected values.

self.assertTrue(prob.result.success)

Example

The following tests do not currently give prob.result.success = True:

test_bench_FwFm.py:TestBenchFwFmSerial.test_bench_FwFm_IPOPT
test_bench_FwGm.py:bench_test_swap_1_GwFm_IPOPT
test_bench_GwFm.py:ProblemPhaseTestCase.bench_test_swap_1_GwFm_IPOPT (doesn't run under testflo runs in PR).
test_FLOPS_detailed_landing.py:TestFLOPSDetailedLanding.bench_test_IPOPT
test_bench_offdesign.py:Test2DOFOffDesign.test_alternate_mission_match SNOPT

These tests are fixed. [KTM]
test_bench_FwFm.py:TestBenchFwFmSerial.test_bench_FwFm_SNOPT exit code 3 requested accuracy not achieved
test_bench_FwGm.py:* (close, but doesn't get a 0,1) [added by KTM]
test_bench_GwFm.py:ProblemPhaseTestCase.bench_test_swap_1_GwFm_SNOPT (doesn't run in testflo or PR, runs in merge queue).
test_FLOPS_based_sizing_N3CC.py:ProblemPhaseTestCase.bench_test_sizing_N3CC exit code 41 current point cannot be improved

These tests no longer exist. Their replacements have the success assert. [KTM]
test_off_design_FwFm.py:TestOffDesign.test_off_design_IPOPT
test_off_design_FwFm.py:TestOffDesign.test_off_design_SNOPT
test_off_design_GwGm.py:TestOffDesign.test_off_design_IPOPT
test_off_design_GwGm.py:TestOffDesign.test_off_design_SNOPT

Aviary Version

0.10.0-dev

Relevant environment information

No response

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.