GEOS-DEV / GEOS-DEV/GEOS

Modification to integrated test tolerance scheme.

Open
#766 11 comments 1 reaction 5 assignees View on GitHub

Nobody has claimed this yet.

type: testing
Dominant language
C++
Stars
287
Forks
109
Avg merge
4d 41m
Merged PRs (30d)
5

Description

Is your feature request related to a problem? Please describe.
Currently, the tolerance scheme is as follows:

Entries x1 and x2 are  considered equal iff
    |x1 - x2| <= ATOL or |x1 - x2| <= RTOL * |x2|.
To measure the degree of difference a scaling factor q is introduced. The goal is now to minimize q such that
    |x1 - x2| <= ATOL * q or |x1 - x2| <= RTOL * |x2| * q.
If RTOL * |x2| > ATOL
    q = |x1 - x2| / (RTOL * |x2|)
else
    q = |x1 - x2| / ATOL.
If the maximum value of q over all the entries is greater than 1.0 then the arrays are considered different and an error message is produced.

This is insufficient to catch diffs across all data that will have different scales. For instance, stress typically has a magnitude on order of 1e6, while displacements have magnitude on order of 1e-3. So the ATOL isn't really applicable for both scales. For example using a RTOL=1e-6, ATOL=1e-6:

For a stress field with a maximum value of 1e6, we can have an entry with x1=1.001, x2=1.0. This will fail RTOL,and fails ATOL. However this is essentially equal to the 9th digit when considering that the maximum value of the field is 1e6. To pass we would need an ATOL>1.0e-3.

In contrast a displacement field with a maximum value of 10e-3 and x1=1.1e-7, x2=1.0e-7. This fails RTOL, but passes ATOL....so these would be considered equal even though there is a pretty large diff when compared to the maximum value of 10e-3.

Describe the solution you'd like

    |x1 - x2| <= ATOL * (1 + max(|x2|) ) 
    or 
    |x1 - x2| <= RTOL * |x2|

where max(\x2\) is the maximum value of all x2 in the problem. This effectively normalizes |x1 - x2| s.t. it may be compared with unity, and thus ATOL will be relative to 1....which essentially is a measure of the number of significant digits we expect the values to have.

With this approach, the previous example would pass for stress, and fail for displacement...which is more appropriate.

Describe alternatives you've considered
Using a really low tolerance.

Additional context
Please suggest any other alternatives.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.