aws / aws/aws-cdk

cli: automatic multi environment integration tests

Open
#32,083 1 comment 0 reactions 0 assignees View on GitHub
effort/medium feature-request p1 package/tools
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the feature

When the CLI deploys a stack as part of an integration test, automatically deploy the stack to multiple environments within a single CLI execution

### Use Case

A single CLI execution can deploy multiple stacks to multiple environments. This opens the door for bugs that arise from code that doesn't take into account that each environment may contain information that affects the deployment of the stack.

For example, code that uses any bootstrap stack resource must not assume that the resource is the same in all environments.

Automatically deploying stacks to multiple environments will catch these bugs without needing to explicitly write a test for it.

### Proposed Solution

There are several technical approaches we can take:

- Provide a base stack to be used in tests which duplicates itself across multiple environments.
- Mutate the assembly at runtime.
- ...

For simplicity from an auth perspective, we can narrow this down to just multi-region tests and not deal with multi account.

Regardless though, since these tests require multiple environments, we'll need to make a change to our region allocator. We might be able to get away with just acquiring an additional region lease here:

https://github.com/aws/aws-cdk/blob/33872f38ca4ef33e05d7f01ab36c192a30ab9137/packages/%40aws-cdk-testing/cli-integ/lib/resource-pool.ts#L69-L76

But since we are planning a revamp to our [environment allocation service](https://github.com/aws/aws-cdk/issues/31938), its likely better to just hold off until we implement it - taking into account requirements surfaced by this use-case.

### Other Information

Some peripheral considerations:

- We need to provide a way for tests to opt out of this behavior (there may be good reasons for it).
- Do we add this as a permutation of the test? or instead?

### Acknowledgements

- [x] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### CDK version used

ALL

### Environment details (OS name and version, etc.)

ALL

Contributor guide

Open the contributing guide

Research direction

Start by reading packages/@aws-cdk-testing/cli-integ/lib/resource-pool.ts around the region lease acquisition and review the environment allocation service work in issue #31938. Determine how integration tests could deploy stacks across multiple regions, support opt-out behavior, and define whether this is a permutation or replacement before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cli, infrastructure, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.