Reduce memory usage of pixel_to_pixel
- Dominant language
- Python
- Stars
- 127
- Forks
- 74
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 2
Description
This isn't really an issue that requires work in reproject but it might require work in several other packages. Basically pixel_to_pixel can be very memory-intensive under certain situations. Here's an example:
https://gist.github.com/astrofrog/a807da0c34b7d669289e3c14d43a099a
this is with a GWCS and regular astropy WCS object. The input arrays are 80Mb each (160Mb total) but the memory peaks around 1.7-1.8Gb which is 10x more than this. It might be there's not much we can do about it, but it'd be interesting to see if there are places where we memory could be used more efficiently. Obviously, the output arrays also need 80Mb each, but that's still a lot of overhead.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the pixel_to_pixel entry point and reproduce the memory profile using the linked gist with GWCS and astropy WCS inputs. Trace where the 80 MB input and output arrays are duplicated, then compare peak memory and output correctness after any change; the issue does not name specific files or a target peak.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100