[EKS] [Fargate] [request]: PTRACE support is incomplete and require matching UID between containers

**Current behavior:**

We could use PTRACE calls and functionality guarded by CAP_SYS_PTRACE only for processes that share UID between containers.
More specifically, the agent is running in a sidecar container (“agent” container) in addition to the other containers (“user” containers) in the pod.
Only when the “user” container shares the same UID as the “agent” container (using securityContext: runAsUser) the PTRACE calls from a process running in the “agent” container are successfully attached to processes running in the “user” container.

From our research, this situation occurs because the processes running in the “agent” container don't have the SYS_PTRACE capability, but when trying to add this capability to the “agent” container, the pod fails to schedule on Fargate because CAP_SYS_PTRACE isn’t supported there.

**Expected behavior:**

Have the ability to use PTRACE in fargate EKS without having to enforce matching UIDs.

**How to reproduce:**
Following are steps and YAMLs for the reproduction of the problem and examples:

General notes:
- We are using `strace` to simulate PTRACE attach functionality and get the output
- We are using UID=0 to be able to use `apk` to install `strace`, the same behavior is demonstrated with any other UID
- A screenshot is attached, upper part is the Fargate pods, lower part is the “regular” node pods

**Reproduction in Fargate EKS environment -** `ptrace_fargate.yaml` (attached):

Defines 3 pods, each with “agent” and “user” containers.
The pods are running in the “fargate-apps” namespace and have labels set up to leverage the pre-defined Fargate profile and run as EKS Fargate pods

1. Pod number 1, fargate-different-uid (tab 1 in the screenshot):
- Each container is running with a different UID
- Strace execution fails - permission denied
- No access to /proc//root - permission denied
- Permission is denied since both calls require the CAP_SYS_PTRACE capability, which isn’t added

2. Pod number 2, fargate-same-uid (tab 3 in the screenshot):
- Both containers are running with the same UID
- Strace executed successfully
- Access to /proc//root is granted

3. Pod number 3, fargate-different-uid-cap-sys-ptrace (tab 5 in the screenshot):
- The pod fails to even schedule in the EKS Fargate
- Using `kubectl events` we can see the reason for the failure:
`Pod not supported on Fargate: invalid SecurityContext fields: Capabilities added: SYS_PTRACE`

**Reproduction in non-fargate EKS environment -** `ptrace_node.yaml `(attached):
Defines 3 pods, each with “agent” and “user” containers.
The pods are running in the default namespace on regular nodes

1. Pod number 1, node-different-uid (tab 2 in the screenshot):
- Each container is running with a different UID
- Strace execution fails - permission denied
- No access to /proc//root - permission denied
- Permission is denied since both calls require the CAP_SYS_PTRACE capability, which isn’t added.
- Note that this is the same behavior as in the Fargate pods and can be considered the expected behavior.

2. Pod number 2, node-same-uid (tab 4 in the screenshot):
- Both containers are running with the same UID
- Strace executed successfully
- Access to /proc//root is granted

3. Pod number 3, node-different-uid-cap-sys-ptrace (tab 6 in the screenshot):
- Contrary to what happens in Fargate, this pod is created successfully
- Each container is running with a different UID
- Strace executed successfully
- Access to /proc//root is granted
- Permission to use PTRACE and access /proc//root is granted because the “agent” container is running with the CAP_SYS_PTRACE capability, which is the expected outcome

This issue is related to #1102

[ptrace_node.txt](https://github.com/aws/containers-roadmap/files/12791434/ptrace_node.txt)
[ptrace_fargate.txt](https://github.com/aws/containers-roadmap/files/12791435/ptrace_fargate.txt)

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

Start with the attached ptrace_fargate.txt and ptrace_node.txt reproductions and compare the three pod definitions in each environment. Review the relationship to issue #1102 and the documented Fargate security-context limitation. Done means PTRACE works across differently assigned container UIDs on EKS Fargate without adding the unsupported SYS_PTRACE capability.

Written by the indexing model from the issue text.