equinor / equinor/oneseismic-api

Floating point arithmetic and precision

Open
#192 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
5
Forks
8
PR merge metrics
No merged PRs in 30d

Description

Our API, as well as OpenVDS's API operate primarily on floats. However, we do a lot of intermediate floating point arithmetic before we pass values along to OpenVDS. E.g. coordinate transformations. To avoid precision loss (accumulating precision loss) we do most of these calculations on double and convert back to float only when passing the values along to OpenVDS. However, [this](https://github.com/equinor/vds-slice/blob/1a921badf51bb6dba35953b82ce0d3189e5aa351/internal/core/cppapi_data.cpp#L362C9-L362C9) annotation to voxel transform computes on floats, and when used [here](https://github.com/equinor/vds-slice/blob/1a921badf51bb6dba35953b82ce0d3189e5aa351/internal/core/axis.hpp#L29) it introduces a precision loss to the surface depths we are fetching from OpenVDS.

To fix this we should do this annotation to voxel transformation on doubles

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.