EKS: ENABLE_POD_ENI causing hung pods on unsupported instance types (t3, t3a)

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
What do you want us to build? There is an issue where, when using security groups for pods and setting ENABLE_POD_ENI to true, unsupported instance types do not get a Trunk ENI, which is expected, but, they still lose an ENI slot, but the scheduler thinks it can still attach an ENI. This leads to pods getting scheduled, but they can never get an IP address.

**Which service(s) is this request for?** EKS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
What outcome are you trying to achieve, ultimately, and why is it hard/impossible to do right now? What is the impact of not having this problem solved? The more details you can provide, the better we'll be able to understand and solve the problem.

The section of code below is what is causing the issue. Ideally unsupported instance types would be ignored so that they can get the correct number of ENIs.

https://github.com/aws/amazon-vpc-cni-k8s/blob/2b91029cf595117b3ebf13dff159e1463449ed20/pkg/ipamd/ipamd.go#L801]

**Are you currently working around this issue?**
How are you currently solving this problem? The currently workaround is to create multiple versions of aws-node, one for supported instance types and one for unsupported instance type. For the regular aws-node, if I wanted t3a.large instances, I would remove these from the default aws-node by adding this to the affiinity.

- key: node.kubernetes.io/instance-type
operator: NotIn
values:
- t3a.large

Then I would use the attached yaml file (aws-node-t3.txt) to create aws-node-t3 daemonset. Then set ENABLE_POD_ENI only on aws-node and not aws-node-t3.

**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.
[aws-node-t3.txt](https://github.com/aws/containers-roadmap/files/7520790/aws-node-t3.txt)
)

Start at pkg/ipamd/ipamd.go#L801 and review the attached aws-node-t3.txt workaround. Trace how ENABLE_POD_ENI handles unsupported instance types and how ENI capacity is reported to the scheduler. Done means unsupported types receive the correct ENI count, pods are scheduled appropriately, and they can obtain IP addresses.

Written by the indexing model from the issue text.