[Fargate] [request]: EKS Fargate not allowing to set system limits on mmap counts require to run Elasticsearch
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
I wish to set the vm.max_map_count in /etc/sysctl.conf on EKS Fargate in order to work around the memory consumption issue while using Elasticsearch. We wanted to move our project deployment on EKS Fargate but one of our microservice have elasticsearch as internal service which fails to launch on EKS fargate due to vm.max_map_count settings.
Do we have any plan to add support to EKS fargate to add/modify node level settings like this.
**Which service(s) is this request for?**
EKS, Fargate
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
I am trying to launch Elasticsearch service on fargate with EFS CSI as persistent storage but as
Elasticsearch uses a mmapfs directory by default to store its indices. The default operating system limits on mmap counts is likely to be too low, which may result in out of memory exceptions.
we can increase the limits by running the following command on node using privileged init container or daemonset
sysctl -w vm.max_map_count=262144
https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html
But as AWS Fargate on EKS neither it supports running privileged containers nor it provides a way to update this settings we could not able to use fargate for our workloads.
**Are you currently working around this issue?**
We are currently use node groups with EC2 instances as there is no possible way to solve this.
**Additional context**
From AWS support case i got to know we have this Active Feature Request on EKS Fargate. Could you tell me what time frame we have. Could you please let us know if we can have any workaround for now to use EKS fargate with this use case.
**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)
Contributor guide
Research direction
Start by reviewing the EKS Fargate and sysctl limitations described in the issue, focusing on vm.max_map_count, privileged containers, and node-level settings. Compare the requested capability with the documented Elasticsearch and EFS CSI requirements; done means identifying whether supported configuration or a Fargate platform change can address this use case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, elasticsearch, kubernetes
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100