Large memory consumptions with large number of input files
Open
- Dominant language
- Go
- Stars
- 131
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
I suspect that the cache map starts consuming a very large amount of memory after a while. I ran it on a machine with 32gb RAM and nothing other than postgres and hashr running and it killed hashr for OOM.
Should we consider adding support for offloading the cache to the database or something like redis?
[hashr-crash3.log](https://github.com/google/hashr/files/11183210/hashr-crash3.log)
Contributor guide
Assessment
This issue has not been assessed yet.