get2knowio / get2knowio/remo

Add configurable memory limits to Incus containers

Open
#19 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2
Forks
0
Avg merge
15h 59m
Merged PRs (30d)
9

Description

## Summary

Incus containers created by `incus_container` role have no `limits.memory` set, meaning a single container can consume all host RAM. This caused a production crash on `lab1` (2026-03-21) when Docker container churn inside a dev container exhausted all 32GB host memory, triggering a btrfs + overlay2 deadlock in `kswapd` that required a hard reboot.

## Root Cause

- `kswapd` (kernel memory reclaimer) tried to free cached dentries
- This triggered `ovl_destroy_inode` (Docker overlay) → `btrfs_evict_inode` → `btrfs_force_cow_block`
- btrfs COW needed to allocate pages, but `kswapd` was already in the `PF_MEMALLOC` reclaim path
- Deadlock: the reclaimer needed memory to free memory
- No swap was configured, so there was zero safety margin

## Proposed Change

Add an optional `incus_container_memory_limit` parameter to the `incus_container` role defaults:

```yaml
# defaults/main.yml
incus_container_memory_limit: "" # e.g., "24GB", empty = no limit
```

When set, apply via:
```yaml
- name: Set memory limit
ansible.builtin.command:
cmd: incus config set {{ incus_container_name }} limits.memory {{ incus_container_memory_limit }}
when: incus_container_memory_limit | length > 0
```

## Additional Context

- Host `lab1` is a 32GB Ryzen 9 mini-PC running openSUSE Tumbleweed with btrfs
- An 8GB swap file has been added as a mitigation
- The dev1 container has been manually set to `limits.memory=24GB`
- The btrfs + Docker overlay2 + no-swap combination is particularly fragile under memory pressure

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.