pyronear / pyronear/pyro-engine
Avoid host crash due to docker excessive resources use
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 19
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
According to docker documentation :
By default, a container has no resource constraints and can use as much of a given resource as the host's kernel scheduler allows.
and this is how pyro-engine actually works.
However, we have recently encounter issues, rpi crash, probably due to a due to excessive use of multiprocessing and it has needed manual reboot. https://github.com/pyronear/pyro-engine/blob/feb54de42c050cc59797e2adbbcabc52d3e8a607/pyroengine/core.py#L4
To prevent this kind of situation from recurring, we could consider limiting usage by docker Memory, CPUs, and GPUs.
I'd like to know the best practices on this subject (which parameters to play with ... hard or soft limits for memory..... if the rpi has 4 CPUs, is relevant to setup a limit to 3.8 .. etc ), which limitations to implement, and whether there are any constraints to implementing this type of docker limitation.
I would tend to think that we want to minimize the limits imposed on the docker service, but make sure we avoid a complete host crash
Thank you in advance for your suggestions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with pyroengine/core.py at the linked line and review Docker's resource constraints documentation. Determine which CPU, memory, and GPU limits are appropriate for the Raspberry Pi deployment and identify implementation constraints; done requires agreed limits and a documented implementation plan.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python, raspberry-pi
- Domain
- devops, embedded-iot, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100