aws / aws/aws-cdk

cdk-cli: allow --no-previous-parameters to be configured in `cdk.json` alongside other config

Open
#26,418 2 comments 9 reactions 0 assignees View on GitHub
@aws-cdk/aws-ssm cli feature-request needs-design needs-discussion p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
1d 19h
Merged PRs (30d)
74

Description

### Describe the feature

When using CDK aspects like `StringParameter.fromStringParameterAttributes` / `StringParameter.valueForStringParameter`; by default, they use CloudFormation Stack Parameters; which are fixed at the point of first deploy, unless we deploy with `cdk deploy --no-previous-parameters`.

This is often confusing and unexpected, as outlined in this issue:

- https://github.com/aws/aws-cdk/issues/7722

### Use Case

I want to enforce `--no-previous-parameters` to always be used in my `cdk.json`, so that I can commit it into my project repo.

### Proposed Solution

Allow `--no-previous-parameters` to be configured in `cdk.json` alongside existing config values.

### Other Information

You can see some of my deepdive comments on the following issue:

- https://github.com/aws/aws-cdk/issues/7722#issuecomment-1637516435
- https://github.com/aws/aws-cdk/issues/7722#issuecomment-1639188041

Notably in that 2nd comment, it appears that some very recent changes (~2 weeks ago) made in CDK `2.87.0` would sort of allow the root issue to be worked around in a different way if I made use of `forceDynamicReference`:

> > **Edit 3:** Looking closer at the CDK docs for `StringParameter.fromStringParameterAttributes(scope, id, attrs)`, and particularly `StringParameterAttributes`, there appears to be a `forceDynamicReference` option:
> >
> > * [docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ssm.StringParameter.html#static-fromwbrstringwbrparameterwbrattributesscope-id-attrs](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ssm.StringParameter.html#static-fromwbrstringwbrparameterwbrattributesscope-id-attrs)
> > * `StringParameterAttributes`: [docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ssm.StringParameterAttributes.html#forcedynamicreference](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ssm.StringParameterAttributes.html#forcedynamicreference)
> >
> > * > Use a dynamic reference as the representation in CloudFormation template level.
> > > By default, CDK tries to deduce an appropriate representation based on the parameter value (a CfnParameter or a dynamic reference). Use this flag to override the representation when it does not work.
>
> Ha.. so apparently `forceDynamicReference` and the underlying functionality related to it is brand new as of CDK `2.87.0` (released ~2 weeks ago):
>
> - https://github.com/aws/aws-cdk/releases/tag/v2.87.0
> - > ssm: cannot import a ssm parameter with a name containing unresolved token ([1f1b642](https://github.com/aws/aws-cdk/commit/1f1b6426f00031dc0f67b9690d33f718f394733c))
> > - https://github.com/aws/aws-cdk/issues/25749
> > - closes https://github.com/aws/aws-cdk/issues/17094
>
> > Previously, when we import a SSM parameter by `ssm.StringParameter.fromStringParameterAttributes`, we use `CfnParameter` to get the value.
> >
> > ```json
> > "Parameters": {
> > "importsqsstringparamParameter": {
> > "Type": "AWS::SSM::Parameter::Value",
> > "Default": {
> > "Fn::ImportValue": "some-exported-value-holding-the-param-name"
> > }
> > },
> > ```
> >
> > However, `Parameters..Default` only allows a concrete string value. If it contains e.g. intrinsic functions, we get an error like this from CFn: `Template format error: Every Default member must be a string.`
> >
> > This PR changes the behavior of `fromStringParameterAttributes` method. Now it uses `CfnDynamicReference` instead of `CfnParameter` if a parameter name contains unresolved tokens.
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749_
>
> > Another thing we can say about ssm parameters is that it doesn't differ much between [CfnParameters](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html#aws-ssm-parameter-types) and [dynamic references](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html). Reading through the document, it seems that most of the characteristics are the same, such as when it's resolved and updated, where it can be used in a template, etc. There are of course some differences e.g. max num of references (200 vs 60), but they seems trivial.
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749#discussion_r1213201913_
>
> > I'm now wondering whether switching a [parameter](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html#aws-ssm-parameter-types) to a [dynamic reference](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html) should really be considered as a breaking change. As far as I read the docs, there seems to be no remarkable difference between them. Given it's also very rare to use a lazy token for parameter names, we can tolerate the change, maybe under a feature flag.
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749#discussion_r1229498664_
>
> > If only we could stop using `CfnParameter` and use `CfnDynamicReference` instead for all cases... I see no point to use `CfnParameter` here. (Actually, isn't that the purpose of feature flags?)
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749#discussion_r1229662611_
>
> > There are some limitations https://github.com/aws/aws-cdk/pull/22239#issuecomment-1262069499
> >
> > _Originally posted by @corymhall in https://github.com/aws/aws-cdk/pull/25749#discussion_r1229669719_
>
> > So how about letting users choose which they use, parameter or dynamic reference? We'll add a property like `forceDynamicReference?: boolean` (default to false) to `CommonStringParameterAttributes`. This is kind of a leaky abstraction, but it should at least solve all the problem above. Plus we can easily ensure there is no breaking change, without adding any feature flag.
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749#discussion_r1229717891_
>
> _Originally posted by @0xdevalias in https://github.com/aws/aws-cdk/issues/7722#issuecomment-1639188041_

---

Then, on an internal issue, I noted the following while researching whether it was possible to configure this in `cdk.json`:

> ## Avoiding this issue coming up again in future
>
> Since I don't believe we really make use of CDK / CloudFormation parameters directly ourselves (and it's apparently not really a recommended thing to do either), I wonder if there would be any harm in us forcing CDK to always update them so that we avoid this issue in future?
>
> Looking in AWS web console -> CloudFormation, for each of our stacks:
>
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter + SSM parameters
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter + SSM parameters
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter
>
> ✅ Based on the above, it looks like it should be safe to always deploy our stacks with `--no-previous-parameters`!
>
> ---
>
> Looking at the `cdk deploy --help`, here are the relevant parts that mention parameters:
>
> ```shell
> ⇒ yarn cdk-dev deploy --help
> yarn run v1.22.19
> $ yarn -s awsvault-dev:nosession cdk deploy --help
> cdk deploy [STACKS..]
>
> Deploys the stack(s) named STACKS into your AWS account
>
> Options:
> ..snip..
>
> -c, --context Add contextual string parameter (KEY=VALUE) [array]
>
> ..snip..
>
> --parameters Additional parameters passed to CloudFormation at
> deploy time (STACK:KEY=VALUE) [array] [default: {}]
> ..snip..
>
> --previous-parameters Use previous values for existing parameters (you
> must specify all parameters on every deployment if
> this is disabled) [boolean] [default: true]
> ..snip..
>
> ✨ Done in 6.61s.
> ```
>
> `--previous-parameters` defaults to `true`, which is why we have to use `--no-previous-parameters` to disable it.
>
> It seems that at least some of the CDK CLI flags can be configured in the project's `cdk.json` file:
>
> - https://docs.aws.amazon.com/cdk/v2/guide/cli.html#cli-config
>
> Unfortunately this doesn't seem to include the `previous-parameters` / `no-previous-parameters` option.
>
> We can confirm that by looking at CDK's `Configuration` class:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/settings.ts#L68-L158
>
> and `Settings` class:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/settings.ts#L232-L445
>
> Neither of which seem to contain entries for `previous-parameters` / `previousParameters`, which only seem to be meaningfully used by:
>
> The `cdk bootstrap` command:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/cli.ts#L510
>
> The `cdk deploy` command:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/cli.ts#L569
>
> And a commented out reference in the `cdk watch` command:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/cli.ts#L603
>
> ---
>
> Based on the above, it seems like our simplest/best option right now might be to just hardcode `--no-previous-parameters` into our deploy helper script in `package.json` or similar (⚠️ TODO: and probably also raise an upstream issue on the AWS CDK repository to explore whether this is something that can be added to the CDK)
>
> _Originally posted by @0xdevalias in https://github.com/PsychNEXUS/REMDR/issues/278#issuecomment-1637500082_

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### CDK version used

2.84.0 (build f7c792f)

### Environment details (OS name and version, etc.)

macOS / N/A

Contributor guide

Open the contributing guide

Research direction

Start in packages/aws-cdk/lib/settings.ts, reviewing Configuration and Settings, then trace how the deploy command uses the option in packages/aws-cdk/lib/cli.ts. Confirm the relevant setting can be configured in cdk.json alongside existing configuration and that cdk deploy honors it.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cli
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.