gpilab / gpilab/philips-data-reader

memory allocation issue

Open
#16 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
14
Forks
5
PR merge metrics
No merged PRs in 30d

Description

Hi, I was trying to get working the ReadPhilips node and I'm getting memory allocation issues, in particular, I get the following error:

`File "ReadPhilips_GPI.py", line 574, in compute
File "readPhilipsExports.py", line 1506, in readRaw
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 97.3 GiB for an array with shape (28, 1, 450, 1, 9, 1, 1, 1, 1, 1, 1, 450, 256) and data type complex64`

So the library is trying to pre-allocate almost a 100GB for the array shown with those dimensions. Clearly, trying to pre-allocate memory for all the dimensions of the array seems kinda unnecessary (dimensions with size 1), they may provide useful information in the reading process but considering all of them in the pre-allocation step seems overkill. Is there anyway to circumvent this issue? It may not even be related directly with the code, maybe it has to do with the .sin file, I don't know, it would be really helpful to get some guidance so I can perform the functionality that I'm looking for.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.