aws / aws/containers-roadmap

[EKS] [request]: Update Windows Nodes ContainerD version

Open
#2,163 11 comments 3 reactions 1 assignee Claimed by @goyalpurvi View on GitHub
EKS Proposed Windows
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
Update EKS Managed Windows nodes to a use a more updated version of Containerd

**Which service(s) is this request for?**
EKS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
_Background:_
We have some Windows solution which we migrated from EC2 machines into a Container format.
The solution requires us to have a graceful shutdown process as it common in any application scenario.
Windows behaves differently when it comes to Signal Termination Handling than Linux.
The equivalent of SigTerm in Windows is CTRL_SHUTDOWN_EVENT.
For more details: Our application is rather old, and is built on Framework 4.7.2 - so our container is Windows2019-Server-Core

_The issue:_
We have an application that requires to process itself to completion and then shutdown safely.
As per Kubernetes documentation you may use the field 'terminationGracePeriodSeconds' which effectively should wait N number of seconds until forcibly killing the container\pod unless the process within exits before the given time.
This Field is honored by Kubelet and Kubernetes but has no affect on the Innards of the container.

Hence we ventured out an found out that you have to specify a Registry key to make Windows wait more time before shutting down the process.
This Key is defaults to 5 seconds - Meaning no matter what you have in Windows - unless changed before Container startup - the windows will shut itself down after 5 seconds.
you can read about it here in this comment - [https://github.com/moby/moby/issues/25982#issuecomment-426441183](https://github.com/moby/moby/issues/25982#issuecomment-426441183)

So we added that key, to a different value - say 30000 ms which is 30 seconds.
However, we discovered that its not honored by EKS - on any version.

_Here's why:_
Following this issue - [https://github.com/microsoft/Windows-Containers/issues/164](https://github.com/microsoft/Windows-Containers/issues/164)

It is noted that the library used in containerd - hcsshim. Doesn't respect that shutdown call but it was implemented in the version 0.9.7 of hcsshim and baked into containerd 1.6 and 1.7.
You can read about it [here](https://github.com/microsoft/Windows-Containers/issues/164)

Digging deeper it was specifically added at earliest version [1.6.19](https://github.com/containerd/containerd/blob/v1.6.19/go.mod)

EKS Windows Core 2019 and 2022 support only version 1.6.6! as specified [here](https://docs.aws.amazon.com/eks/latest/userguide/eks-ami-versions-windows.html#eks-ami-versions-windows-2019-core)

In essence this means that EKS doesn't natively support any GracefulShutdown for windows containers.

**Are you currently working around this issue?**
- One of the options we are trying is to add a "sleeper" script which is prebaked with our container and use the Lifecycle PreStop in Kubernetes but this doesn't guarantee 100% graceful shutdown.
- Currently I'm attempting to create my own EKS Windows Optimized AMI with a more updated version of Containerd

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.