cdklabs / cdklabs/cdk-pipelines-github

Bug/Question: Why is stepsToDownloadAssembly() called with cdkoutDir?

Open
#706 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
384
Forks
45
Avg merge
1d 1h
Merged PRs (30d)
4

Description

**Problem**

We're trying to leverage this library to run a few pipelines in our CDK project, and we were running into problems where each time the `npx projen build` was run, we got a modified `deploy.yml` file. Each and every time we ran it, the diff changes:

```diff
% npx projen build && git diff .github/workflows/deploy.yml
...
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
index f8eb654..2a6ed0e 100644
--- a/.github/workflows/deploy.yml
+++ b/.github/workflows/deploy.yml
@@ -49,7 +49,7 @@ jobs:
uses: actions/download-artifact@v3
with:
name: cdk.out
- path: /private/var/folders/dm/b5by_qw91nd0ctdjbvggzgr40000gq/T/cdk.outz8GheM
+ path: /private/var/folders/dm/b5by_qw91nd0ctdjbvggzgr40000gq/T/cdk.outiCdOwR
- name: Install
run: npm install --no-save cdk-assets
- name: Authenticate Via OIDC Role
% npx projen build && git diff .github/workflows/deploy.yml
...
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
index f8eb654..2a6ed0e 100644
--- a/.github/workflows/deploy.yml
+++ b/.github/workflows/deploy.yml
@@ -49,7 +49,7 @@ jobs:
uses: actions/download-artifact@v3
with:
name: cdk.out
- path: /private/var/folders/dm/b5by_qw91nd0ctdjbvggzgr40000gq/T/cdk.outz8GheM
+ path: /private/var/folders/dm/b5by_qw91nd0ctdjbvggzgr40000gq/T/cdk.outBt0txL
- name: Install
run: npm install --no-save cdk-assets
- name: Authenticate Via OIDC Role
```

Digging through the code, I found that [`stepsToDownloadAssembly()`](https://github.com/cdklabs/cdk-pipelines-github/blob/e1d219e54d9c32b7759c6f5079b9998b3aebe708/src/pipeline.ts#L811-L824) is being called with [`targetDir: cdkoutDir`](https://github.com/cdklabs/cdk-pipelines-github/blob/e1d219e54d9c32b7759c6f5079b9998b3aebe708/src/pipeline.ts#L539C19-L539C42). Tracing the `cdkoutDir` back, we find that it's basically set to `app.outdir` at https://github.com/cdklabs/cdk-pipelines-github/blob/e1d219e54d9c32b7759c6f5079b9998b3aebe708/src/pipeline.ts#L357.

**What does this cause?**

Each and every time the `build.yml`'s `self-mutation` step runs, it detects a change to `.github/workflows/deploy.yml` and commits that change, causing a build loop (presuming you have granted your Projen Github App with the right credentials so that it can mutate the workflow files).

**Our fix: Set CDK_OUTDIR**

If we explicitly set the `CDK_OUTDIR` environment variable to `/tmp`, we can convince the system to generate a consistent `deploy,yml` file:

```diff
% npx projen build && git diff .github/workflows/deploy.yml
...
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
index f8eb654..2a6ed0e 100644
--- a/.github/workflows/deploy.yml
+++ b/.github/workflows/deploy.yml
@@ -49,7 +49,7 @@ jobs:
uses: actions/download-artifact@v3
with:
name: cdk.out
- path: /private/var/folders/dm/b5by_qw91nd0ctdjbvggzgr40000gq/T/cdk.outz8GheM
+ path: /tmp
- name: Install
run: npm install --no-save cdk-assets
- name: Authenticate Via OIDC Role
```

The trick was, how to do this programmatically? So we added a few hacks to our `.projenrc.ts` file:

```typescript
# .projenrc.ts
project.github?.tryFindWorkflow('build')?.file?.addOverride('env', { CDK_OUTDIR: '/tmp' });
```

```typescript
# main.ts
const pipeline = new GitHubWorkflow(app, 'StagingPipeline', {
...
synth: new ShellStep('Build', {
commands: ['yarn install', 'yarn build'],
env: {
CDK_OUTDIR: '/tmp',
},
}),
```

**Question: Is this an intentional design decision?**

My main question here is whether or not it is an intentional decision that the `deploy.yml` file would be mutated each and every time that `npx projen build` is run? That seems pretty strange .. yet I can't find any evidence of other users complaining about this.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.