equinor / equinor/resdata

Accessing grdecl grid cells without allocating memory for the whole file?

Open
#720 1 comment 0 reactions 0 assignees View on GitHub
enhancement performance
Dominant language
C++
Stars
127
Forks
103
Avg merge
2d 5h
Merged PRs (30d)
18

Description

I tried libecl to access grdecl files from C/C++.

From grdecl files, i need to extract the cell corners coordinates, to find wether a cell is present or not, and the values for some properties for each cell. But i am having issues of excessive memory consumption. Trying to open a 4GB grdecl file is crashing my computer (16GB RAM) with too big allocs (grid ix about 500x500x500).

What i do is
- Load the various properties using ecl_kw_fscanf_alloc_grdecl_dynamic__(). This stores values in binary format, but uncompresses the '5*-9999' elements, so the buffer for the COORD, ACTNUM, & ZCORN is still 4GB in memory
- call ecl_grid_alloc_GRDECL_kw() to create an ecl_grid_type. This crashes the computer for the big file.

As i understand, this will always allocate a buffer of nx*ny*nz cells. And the structure for a cell is about 250 bytes each. For a 500x500x500 grid, this only already needs 31GB or RAM.

I don't even need to have all cells in memory at a given time. I could just go with "cell iterator", that from raw ecl_kw_type buffers, gives me access to the corner coordinates for each cell sucesssive cell, so i can store them in an other format.
Is there a way to access the cells geometry information and properties values without the need to have this much data in memory?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.