aws / aws/aws-cdk

pipelines: fix integ tests before we run them automatically

Open
#28,618 0 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/integ-tests @aws-cdk/pipelines bug effort/medium p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
1d 19h
Merged PRs (30d)
74

Description

### Describe the bug

We have a few tests in pipelines that, because of a limitation, reference specific repositories that are not under the `aws-cdk` scope. This is not an urgent issue for us because we do not run these integ tests automatically. However, it is less than ideal and a blocker if we do decide to run automatic integ tests.

To mitigate this issue, we need to replace instances of `huijbers/cdk-pipelines-demo` and other specific repositories with a parameter that can be provided via environment variable. This means that the integ test cannot succeed manually without providing this variable, and that we would have to `--dry-run` with a dummy value when committing the integ test snapshot into the repo.

This should be a more or less exhaustive list, plus the test in #28447

```bash
❯ rg cdk-pipelines-demo

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline.js.snapshot/PipelineStackPipeline9DB740AF.dot
10:"Source.rix0rrr/cdk-pipelines-demo" -> "Build.Synth";
189:"Source.rix0rrr/cdk-pipelines-demo";
190:"BEGIN Source" -> "Source.rix0rrr/cdk-pipelines-demo";
191:"Source.rix0rrr/cdk-pipelines-demo" -> "END Source";

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline.js.snapshot/tree.json
343: "name": "rix0rrr_cdk-pipelines-demo",
357: "Repo": "cdk-pipelines-demo",
1953: "rix0rrr_cdk-pipelines-demo": {
1954: "id": "rix0rrr_cdk-pipelines-demo",
1955: "path": "PipelineStack/Pipeline/Pipeline/Source/rix0rrr_cdk-pipelines-demo",
1959: "path": "PipelineStack/Pipeline/Pipeline/Source/rix0rrr_cdk-pipelines-demo/WebhookResource",
1973: "targetAction": "rix0rrr_cdk-pipelines-demo",

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline.js.snapshot/manifest.json
115: "/PipelineStack/Pipeline/Pipeline/Source/rix0rrr_cdk-pipelines-demo/WebhookResource": [

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline.js.snapshot/PipelineStack.template.json
253: "Repo": "cdk-pipelines-demo",
258: "Name": "rix0rrr_cdk-pipelines-demo",
1864: "TargetAction": "rix0rrr_cdk-pipelines-demo",

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline.ts
14: input: pipelines.CodePipelineSource.gitHub('rix0rrr/cdk-pipelines-demo', 'main'),

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-account-keys.ts
16: input: pipelines.CodePipelineSource.gitHub('tkglaser/cdk-pipelines-demo', 'main'),

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-region-replication-buckets.ts
51: 'jose-clickup/cdk-pipelines-demo',

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-region-replication-buckets.js.snapshot/PipelineStack.template.json
161: "Repo": "cdk-pipelines-demo",
166: "Name": "jose-clickup_cdk-pipelines-demo",
468: "TargetAction": "jose-clickup_cdk-pipelines-demo",

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-region-replication-buckets.js.snapshot/PipelineStackPipeline9DB740AF.dot
10:"Source.jose-clickup/cdk-pipelines-demo" -> "Build.Synth";
52:"Source.jose-clickup/cdk-pipelines-demo";
53:"BEGIN Source" -> "Source.jose-clickup/cdk-pipelines-demo";
54:"Source.jose-clickup/cdk-pipelines-demo" -> "END Source";

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-region-replication-buckets.js.snapshot/manifest.json
239: "/PipelineStack/Pipeline/Pipeline/Source/jose-clickup_cdk-pipelines-demo/WebhookResource": [

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-region-replication-buckets.js.snapshot/tree.json
807: "name": "jose-clickup_cdk-pipelines-demo",
821: "Repo": "cdk-pipelines-demo",
1112: "jose-clickup_cdk-pipelines-demo": {
1113: "id": "jose-clickup_cdk-pipelines-demo",
1114: "path": "PipelineStack/Pipeline/Pipeline/Source/jose-clickup_cdk-pipelines-demo",
1118: "path": "PipelineStack/Pipeline/Pipeline/Source/jose-clickup_cdk-pipelines-demo/WebhookResource",
1133: "targetAction": "jose-clickup_cdk-pipelines-demo",

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-account-keys.js.snapshot/manifest.json
127: "/PipelineStack/Pipeline/Pipeline/Source/tkglaser_cdk-pipelines-demo/WebhookResource": [

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-account-keys.js.snapshot/PipelineStack.template.json
358: "Repo": "cdk-pipelines-demo",
363: "Name": "tkglaser_cdk-pipelines-demo",
1978: "TargetAction": "tkglaser_cdk-pipelines-demo",

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-account-keys.js.snapshot/tree.json
480: "name": "tkglaser_cdk-pipelines-demo",
494: "Repo": "cdk-pipelines-demo",
2099: "tkglaser_cdk-pipelines-demo": {
2100: "id": "tkglaser_cdk-pipelines-demo",
2101: "path": "PipelineStack/Pipeline/Pipeline/Source/tkglaser_cdk-pipelines-demo",
2105: "path": "PipelineStack/Pipeline/Pipeline/Source/tkglaser_cdk-pipelines-demo/WebhookResource",
2119: "targetAction": "tkglaser_cdk-pipelines-demo",

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-cross-account-keys.js.snapshot/PipelineStackPipeline9DB740AF.dot
10:"Source.tkglaser/cdk-pipelines-demo" -> "Build.Synth";
189:"Source.tkglaser/cdk-pipelines-demo";
190:"BEGIN Source" -> "Source.tkglaser/cdk-pipelines-demo";
191:"Source.tkglaser/cdk-pipelines-demo" -> "END Source";

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-codebuild-logging.js.snapshot/PipelineStackPipeline9DB740AF.dot
10:"Source.colifran/cdk-pipelines-demo" -> "Build.Synth";
189:"Source.colifran/cdk-pipelines-demo";
190:"BEGIN Source" -> "Source.colifran/cdk-pipelines-demo";
191:"Source.colifran/cdk-pipelines-demo" -> "END Source";

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-codebuild-logging.js.snapshot/tree.json
359: "name": "colifran_cdk-pipelines-demo",
373: "Repo": "cdk-pipelines-demo",
1969: "colifran_cdk-pipelines-demo": {
1970: "id": "colifran_cdk-pipelines-demo",
1971: "path": "PipelineStack/Pipeline/Pipeline/Source/colifran_cdk-pipelines-demo",
1975: "path": "PipelineStack/Pipeline/Pipeline/Source/colifran_cdk-pipelines-demo/WebhookResource",
1989: "targetAction": "colifran_cdk-pipelines-demo",

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-codebuild-logging.js.snapshot/manifest.json
115: "/PipelineStack/Pipeline/Pipeline/Source/colifran_cdk-pipelines-demo/WebhookResource": [

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-codebuild-logging.js.snapshot/PipelineStack.template.json
253: "Repo": "cdk-pipelines-demo",
258: "Name": "colifran_cdk-pipelines-demo",
1864: "TargetAction": "colifran_cdk-pipelines-demo",

packages/@aws-cdk-testing/framework-integ/test/pipelines/test/integ.newpipeline-with-codebuild-logging.ts
20: input: pipelines.CodePipelineSource.gitHub('colifran/cdk-pipelines-demo', 'main'),
```

### Expected Behavior

No references to random repos

### Current Behavior

References to random repos.

### Reproduction Steps

See list

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.x

### Framework Version

_No response_

### Node.js Version

20

### OS

Mac

### Language

TypeScript

### Language Version

_No response_

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with `rg cdk-pipelines-demo` and the listed TypeScript integ tests under `packages/@aws-cdk-testing/framework-integ/test/pipelines/test/`, then inspect their corresponding snapshots. Trace how the repository value can be supplied through an environment variable, and use `--dry-run` with a dummy value when updating snapshots. Done means the integ tests no longer reference random repositories and the snapshots match the parameterized setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
ci-cd, cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.