cockroachdb / cockroachdb/cockroach
operation/failureinjection: OOM
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
There are many failure injection techniques to induce an OOM. E.g., stress-ng, cgroup memory.max, and GOMEMLIMIT. However, these techniques are _out-of-process_, making them somewhat synthetic and side-effect free. E.g.,
```
sudo bash -c "echo -1000 > /proc/self/oom_score_adj && exec stress-ng --vm 2 --vm-bytes 20G --vm-keep --vm-populate --vm-locked --timeout 0 --no-oom-adjust"
```
induces memory pressure and makes itself immune to the OOM killer. It will most likely cause the crdb process to OOM; you can adjust `--vm-bytes` after consulting `free -h`.
Instead, using an _in-process_ method to induce OOM is arguably more realistic. It is likely to have other side-effects. E.g., a resumable job which induces an OOM could affect availability of several nodes.
Jira issue: CRDB-52023
Contributor guide
Research direction
No files, tests, or entry points are named. Start by reading the issue and linked Jira issue CRDB-52023, then compare the cited out-of-process techniques with the requested in-process OOM approach; done should be a defined, realistic failure-injection method with its side effects and scope understood.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100