avast / avast/retdec

Memory limiting does not work on macOS

Open
#379 1 comment 0 reactions 0 assignees View on GitHub
bug C-utils D-help-wanted O-macos
Dominant language
C++
Stars
8.6k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

The limiting of memory via `--max-memory` does not work properly on macOS. For example, if you run
```
$ retdec-fileinfo --max-memory=4096 FILE
```
the analysis (properly) fails on Linux and Windows, but finishes successfully on macOS. The reason is that on macOS, the following piece of code from `src/utils/memory.cpp` compiles, runs, returns 0 (success), but does actually not do anything:
```cpp
struct rlimit rl = {
.rlim_cur = limit, // Soft limit.
.rlim_max = RLIM_INFINITY // Hard limit (ceiling for rlim_cur).
};
auto rc = setrlimit(RLIMIT_AS, &rl);
```
From what I was able to find, `setrlimit()` with `RLIMIT_AS` is broken on macOS and simply does not work.

What is the proper way of programmatically limiting virtual memory on macOS? Feel free to submit a [PR](https://github.com/avast-tl/retdec/pulls) that fixes the implementation of `limitSystemMemoryOnMacOS()` in `src/utils/memory.cpp`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in src/utils/memory.cpp at limitSystemMemoryOnMacOS() and reproduce the issue with retdec-fileinfo --max-memory=4096 FILE on macOS. Compare the macOS behavior with the working Linux and Windows implementations, then verify that the implementation actually limits virtual memory and causes the analysis to fail when the limit is exceeded.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.