test_compliance.py fails due to tolerance assertion. assert np.allclose(final_cost_func, result, rtol=1e-5)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
I have installed Firedrake, PETSC, etc.
The pyMMAopt test_compilance.py fails because np is undefined (python code is missing import numpy as np).
I then discovered the tolerance was too tight for the solution in the test_compliance.py
My question, how can I resolve the compliance testing with rtol=1e-5 ?
What is causing the divergence? Is it HW/SW ? Is there some user parameter I can fiddle with to effect this behavior.
Could it be a compiler optimization flag, or some other setting outside of pyMMAopt? Some setting in PETSC or firedrake itself?
assert np.allclose(final_cost_func, result, rtol=1e-5)
The failure:
(firedrake) [l1057678@rdhpc-n1 tests]$ python3 test_compliance.py
firedrake:WARNING OMP_NUM_THREADS is not set or is set to a value greater than 1, we suggest setting OMP_NUM_THREADS=1 to improve performance
# DOFS: 6262
Volume for MMA is: 29.99999999999996
Value: 3.0000000000000036, Constraint 14.999999999999797
rho0: 315.40116948173716, rhoi: [0.00666667]
Value: 7.872595385768886, Constraint 14.999999999999797
condition: fapp -8755.864348772528, new_fval 131.2644884640787
Recalculating rho
...
It: 21, obj: 7.3906379652341245 g[0]: -0.0007441687698033217
kkt: 0.123831 change: 0.304477 rel obj change: 0.005939
Time per iteration: 0.4173157215118408
Optimization finished with change: 0.30448 and iterations: 21
Traceback (most recent call last):
File "test_compliance.py", line 134, in
test_compliance("L2", 7.420380654729631)
File "test_compliance.py", line 126, in test_compliance
assert np.allclose(final_cost_func, result, rtol=1e-5)
NameError: name 'np' is not defined
Adding in the missing import numpy as np
We then find:
firedrake) [l1057678@rdhpc-n1 tests]$ python3 test_compliance.py
firedrake:WARNING OMP_NUM_THREADS is not set or is set to a value greater than 1, we suggest setting OMP_NUM_THREADS=1 to improve performance
# DOFS: 6262
Volume for MMA is: 29.99999999999996
Value: 3.0000000000000036, Constraint 14.999999999999797
rho0: 315.40116948173716, rhoi: [0.00666667]
Value: 7.872595385768886, Constraint 14.999999999999797
condition: fapp -8755.864348772528, new_fval 131.2644884640787
Recalculating rho
…
It: 21, obj: 7.3906379652341245 g[0]: -0.0007441687698033217
kkt: 0.123831 change: 0.304477 rel obj change: 0.005939
Time per iteration: 0.5018701553344727
Optimization finished with change: 0.30448 and iterations: 21
Traceback (most recent call last):
File "test_compliance.py", line 134, in
test_compliance("L2", 7.420380654729631)
File "test_compliance.py", line 127, in test_compliance
assert np.allclose(final_cost_func, result, rtol=1e-5)
AssertionError
I then experimented with changing the tolerance from 1e-5 -> 1e-2
#assert np.allclose(final_cost_func, result, rtol=1e-5)
assert np.allclose(final_cost_func, result, rtol=1e-2)
(firedrake) [l1057678@rdhpc-n1 tests]$ python3 test_compliance.py
firedrake:WARNING OMP_NUM_THREADS is not set or is set to a value greater than 1, we suggest setting OMP_NUM_THREADS=1 to improve performance
# DOFS: 6262
Volume for MMA is: 29.99999999999996
Value: 3.0000000000000036, Constraint 14.999999999999797
rho0: 315.40116948173716, rhoi: [0.00666667]
Value: 7.872595385768886, Constraint 14.999999999999797
condition: fapp -8755.864348772528, new_fval 131.2644884640787
Recalculating rho
…
It: 21, obj: 7.3906379652341245 g[0]: -0.0007441687698033217
kkt: 0.123831 change: 0.304477 rel obj change: 0.005939
Time per iteration: 0.438732385635376
Optimization finished with change: 0.30448 and iterations: 21
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with test_compliance.py around lines 126-134 and run it in the reported Firedrake and PETSc environment. Check the missing NumPy import, then compare final_cost_func with the expected result and investigate why rtol=1e-5 fails. Done means the cause is identified and the compliance test passes with a justified tolerance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100