aws / aws/containers-roadmap

[EKS] [request]: Anomaly with containerd 2.1.2

Open
#2,791 0 comments 0 reactions 0 assignees View on GitHub
EKS Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

Can find this error occuring with the pods stuck in "CreateContainerError" state for AMI release versions that have containerd version running at 2.2.1 where secrets are not encoded in base64 or in some other binary format:

````
Error: Error: grpc: error unmarshalling request: string field contains invalid UTF-8
````

Example secret file:

````
apiVersion: v1
kind: Secret
metadata:
name: utf8-error-secret
type: Opaque
data:
# This contains invalid UTF-8 when base64 decoded
config: a2V5PXZhbHVlgIH/
````
AMI details:

````
AMI: ami-0c8ded6af22dcae07
amazon-eks-node-al2023-x86_64-standard-1.32-v20260317
````

Corrected secret file:

````
apiVersion: v1
kind: Secret
metadata:
name: utf8-error-secret
type: Opaque
data:
config: YTJWNVBYWmhiSFZsZ0lILw==
````

Also could find that this was not occuring on AMIs that are not running containerd version 2.1.5.

Ask:

Please update this somewhere in the documentation or release notes to support users who upgrade AMI release version or change the AMI ID that upgrades the containerd version too.

Contributor guide

Open the contributing guide

Research direction

Start by verifying the AMI and containerd version details given in the issue, including the differing 2.1.2, 2.2.1, and 2.1.5 references. Find the relevant EKS documentation or release-notes entry, then document the invalid UTF-8 secret behavior and affected AMI upgrades; done when users can identify and avoid the incompatibility.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
documentation, infrastructure
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.