[ECS] [Request]: Fault injection for dependencies of an ECS application

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**

*Please note: I work at ECS and I am soliciting feedback on this proposal.*

I'd like to be able to simulate errors (ex. 503 internal errors, 400 throttling errors) or high latency from my dependent services. I'd like to be able to do this via an integration of [Fault Injection Simulator](https://docs.aws.amazon.com/fis/) and ECS so that I can set up experiments which target specific dependencies and inject errors in the traffic.

An example: I have a service that talks to ECS service X (endpoint `http://svc-x.local:8080`) as well as DynamoDB (endpoint `https://dynamodb.us-west-2.amazonaws.com`). I'd like to know how my service behaves if either of those dependencies throws HTTP 503 errors. For this, I'd like to be able to set up an experiment on FIS that looks like something like this (DynamoDB example):

```
aws fis create-experiment-template \
--cli-input-json '{
"description": "Inject DynamoDB API Internal Errors on 30% of requests",
"targets": {
"AppService": {
"resourceType": "aws:ecs:service",
"resourceArns": [
"arn:aws:ecs:::service/MyCluster/app"
],
"selectionMode": "ALL"
}
},
"actions": {
"InternalError": {
"actionId": "aws:fis:inject-ecs-dependency-failure",
"description": "Inject DynamoDB errors for 30 minutes at the rate of 30%",
"parameters": {
"duration": "PT30M",
"endpoint": "https://dynamodb.us-west-2.amazonaws.com:443",
"percentage": "30",
"httpErrorCode": 503,
"httpResponseHeaders: {"AWS.ErrorCode": "InternalFailure"}
"httpErrorBody": "Service Unavailable"
},
"targets": {
"EcsService": "AppService"
}
}
},
"stopConditions": [
{
"source": "none"
}
],
"roleArn": "arn:aws:iam:::role/service-role/AWSFISIAMRole-ECS",
"experimentOptions": {
"accountTargeting": "single-account",
"emptyTargetResolutionMode": "fail"
}
}'
```

This would generate a 30% fault rate from calls to DynamoDB for 30 minutes.

**Which service(s) is this request for?**
Fargate, ECS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
My application relies on multiple microservices within ECS and dependencies within AWS (ex. S3, DynamoDB). There are a few use cases where fault injection would be beneficial:

1. To understand how the system as a whole behaves when a specific Service sees failures (errors or latency) from its dependencies. Does it result in a retry storm, does it result in unavailability for the calling service etc.
2. To test the resilience of a system by injecting periodic faults from dependencies and ensuring the system as a whole behaves correctly.

**Are you currently working around this issue?**
There is no ECS-native way to solve this. EKS customers are able to use [ChaosMesh](https://chaos-mesh.org/) with FIS as a control plane for injecting faults. AWS AppMesh or ECS Service Connect could help solve this given their usage of an Envoy proxy, but such capabilities do not exist today. In addition, we do not want to migrate to using one of those services just for fault injection

**Additional context**

I'd like this to work for services that support HTTP, HTTP/2 and gRPC and preferably for services over TLS. I'd also like to hear feedback from other ECS customers on how they're working around this issue or if this would solve any additional use cases for you.

Start by reviewing the AWS Fault Injection Simulator integration described in the issue and the example aws fis create-experiment-template configuration for ECS services. Done would mean a documented or implemented way to inject configurable HTTP errors or latency into ECS/Fargate dependency traffic, including the requested HTTP, HTTP/2, gRPC, and TLS cases.

Written by the indexing model from the issue text.