aws / aws/containers-roadmap

[Fargate] [request]: EKS Fargate not allowing to set system limits on mmap counts require to run Elasticsearch

Open
#1,452 4 comments 17 reactions 0 assignees View on GitHub
EKS Fargate Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
I wish to set the vm.max_map_count in /etc/sysctl.conf on EKS Fargate in order to work around the memory consumption issue while using Elasticsearch. We wanted to move our project deployment on EKS Fargate but one of our microservice have elasticsearch as internal service which fails to launch on EKS fargate due to vm.max_map_count settings.

Do we have any plan to add support to EKS fargate to add/modify node level settings like this.

**Which service(s) is this request for?**
EKS, Fargate

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
I am trying to launch Elasticsearch service on fargate with EFS CSI as persistent storage but as

Elasticsearch uses a mmapfs directory by default to store its indices. The default operating system limits on mmap counts is likely to be too low, which may result in out of memory exceptions.

we can increase the limits by running the following command on node using privileged init container or daemonset

sysctl -w vm.max_map_count=262144

https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html

But as AWS Fargate on EKS neither it supports running privileged containers nor it provides a way to update this settings we could not able to use fargate for our workloads.

**Are you currently working around this issue?**
We are currently use node groups with EC2 instances as there is no possible way to solve this.

**Additional context**
From AWS support case i got to know we have this Active Feature Request on EKS Fargate. Could you tell me what time frame we have. Could you please let us know if we can have any workaround for now to use EKS fargate with this use case.
**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the EKS Fargate and sysctl limitations described in the issue, focusing on vm.max_map_count, privileged containers, and node-level settings. Compare the requested capability with the documented Elasticsearch and EFS CSI requirements; done means identifying whether supported configuration or a Fargate platform change can address this use case.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, elasticsearch, kubernetes
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.